Failed assertion in ggml-metal.m

[bjnortier]

I have been testing the SwiftUI example app and I'm getting the following error in ggml-metal.m when running on iPhone and Mac:

-[MTLDebugComputeCommandEncoder setThreadgroupMemoryLength:atIndex:]:817: failed assertion `length(4) must be a multiple of 16 bytes.'

at this line:
https://github.com/ggerganov/whisper.cpp/blob/0463028bc2a5774fe7361c8ac37bef440725bcd7/ggml-metal.m#L1033

since nth is 32 and 32/32*4=4 is not a multiple of 16.

If I remove the /32 it works but I'm not sure what the correct value should be.

Also, running the app without the debugger attached doesn't seem to generate a problem (the assertion doesn't fail).

[iandundas]

Thanks, I was just about to post this too!

[ggerganov]

There is a fix in llama.cpp : ggml-org/llama.cpp#3938

Will sync it here tomorrow

[bjnortier]

Just FYI to anyone else reading this I see the reason why it doesn't fail on Simulator is because use_gpu is false.

#if targetEnvironment(simulator)
        params.use_gpu = false
        print("Running on the simulator, using CPU")
#endif