Eval bug:

[parandandrd]

Name and Version

version: 4606 (a83f528)
built with Apple clang version 16.0.0 (clang-1600.0.26.6) for arm64-apple-darwin24.2.0

Operating systems

Mac

GGML backends

Metal

Hardware

Apple M4 Pro 16-Core GPU

Models

WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B-Q6_K.gguf
Wayfarer-12B-Q6_K.gguf

Problem description & steps to reproduce

Building version 4606 on Mac using default build instructions.

Attempting to run any model results in the error documented in the logs.

First Bad Commit

No response

Relevant log output

Error Domain=MTLLibraryErrorDomain Code=3 "program_source:61:35: error: unknown type name 'block_q4_0'
void dequantize_q4_0(device const block_q4_0 * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:80:38: error: unknown type name 'block_q4_0'
.
.
(lots of similar errors)
.
.
ggml_backend_metal_device_init: error: failed to allocate context
llama_init_from_model: failed to initialize Metal backend
common_init_from_params: failed to create context with model '../../Models/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B-Q6_K.gguf'
main: error: unable to load model

[dannoll]

Hi @parandandrd — did you ever resolve this Metal backend compile issue?

I’m running into what looks like the same problem on an M3 Max (toolchain confirmed, clean environment, working xcrun metal, etc.). The shader fails to compile due to missing type definitions in ggml-metal.metal, like block_q4_0.

Would love to hear if you found a workaround or if it was something on your end. Thanks!

[dannoll]

Anyone encountering similar errors on certain Macbook M1/M2/M3 configurations, you may want to try with the following flag in the configuration step: -DGGML_METAL_EMBED_LIBRARY=OFF. Metal will still be enabled and layers will still be offloaded to GPU, but the build process will use a Metal library external to the binary at runtime.

cmake -B build -DGGML_METAL=ON -DGGML_METAL_EMBED_LIBRARY=OFF
cmake --build build --config Release

[dannoll]

The root cause of this issue (for me, possibly for others): for embedded Metal builds, the build script was mishandling project directory paths that contained spaces. Here's the fix, now merged: #14126