Skip to content

cuda : fix vmm pool with multi GPU #4620

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Dec 26, 2023
Merged

cuda : fix vmm pool with multi GPU #4620

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
32dc09a
cuda : fix vmm pool with multi GPU
slaren Dec 24, 2023
2c3fbf9
hip
slaren Dec 24, 2023
a76cada
use recommended granularity instead of minimum
slaren Dec 24, 2023
6f35a4a
better error checking
slaren Dec 24, 2023
1659cd1
fix mixtral
slaren Dec 25, 2023
865d042
use cudaMemcpy3DPeerAsync
slaren Dec 25, 2023
32304d7
use cuda_pool_alloc in ggml_cuda_op_mul_mat
slaren Dec 25, 2023
692887f
consolidate error checking in ggml_cuda_set_device
slaren Dec 25, 2023
561f1f9
remove unnecessary inlines
slaren Dec 25, 2023
0dcc1a7
style fixes
slaren Dec 25, 2023
23c6dd6
only use vmm for the main device
slaren Dec 25, 2023
da9fc77
fix scratch buffer size, re-enable vmm pool for all devices
slaren Dec 26, 2023
f097bed
remove unnecessary check id != g_main_device
slaren Dec 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading