Skip to content

Trying to split model with --split-max-size, but gguf-split ignores it #6654

@RichardErkhov

Description

@RichardErkhov

Latest version, ubuntu 2204, conda python=3.10.
Trying to split model with gguf-split, but something is going wrong

(base) richard@richard-ProLiant-DL580-Gen9:~/Desktop/ramdisk/banana/llama.cpp$ ./gguf-split --split --split-max-size 4000M --dry-run /media/richard/5fbd0bfa-8253-4803-85eb-80a13218a927/grok-1-fp16-gguf/grok-1-Q5_K.gguf Q5_K/grok-1 
n_split: 1
split 00001: n_tensors = 2115, total_size = 214437M
gguf_split: 1 gguf split written with a total of 2115 tensors.
(base) richard@richard-ProLiant-DL580-Gen9:~/Desktop/ramdisk/banana/llama.cpp$ ./gguf-split --split --split-max-size 4G --dry-run /media/richard/5fbd0bfa-8253-4803-85eb-80a13218a927/grok-1-fp16-gguf/grok-1-Q5_K.gguf Q5_K/grok-1 
n_split: 17
split 00001: n_tensors = 128, total_size = 14609M
split 00002: n_tensors = 128, total_size = 13184M
split 00003: n_tensors = 128, total_size = 12648M
split 00004: n_tensors = 128, total_size = 12597M
split 00005: n_tensors = 128, total_size = 12648M
split 00006: n_tensors = 128, total_size = 12750M
split 00007: n_tensors = 128, total_size = 12836M
split 00008: n_tensors = 128, total_size = 13088M
split 00009: n_tensors = 128, total_size = 13197M
split 00010: n_tensors = 128, total_size = 12597M
split 00011: n_tensors = 128, total_size = 12597M
split 00012: n_tensors = 128, total_size = 12699M
split 00013: n_tensors = 128, total_size = 12699M
split 00014: n_tensors = 128, total_size = 12597M
split 00015: n_tensors = 128, total_size = 13137M
split 00016: n_tensors = 128, total_size = 13675M
split 00017: n_tensors = 67, total_size = 6868M
gguf_split: 17 gguf split written with a total of 2115 tensors.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingsplitGGUF split model sharding

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions