Compile bug: Vulkan shaders not compiling any more on Debian Stable (12/bookworm)

[mbaudier]

Git commit

$ git rev-parse HEAD
5437d4a

Operating systems

Linux

GGML backends

Vulkan

Problem description & steps to reproduce

On an up-to-date Debian Bookworm, Vulkan shaders do not compile any more. After some digging, this seems to be related to changes introduced in b4280 (3df784b) where the GL_KHR_cooperative_matrix extension is being used.

Without being familiar with Vulkan, my understanding is that these extensions started to be introduced with Vulkan 1.3.255 (see https://www.phoronix.com/news/Vulkan-1.3.255), but Vulkan on the current Debian Stable has version 1.3.239. Here are the versions of various packages which may be related:

$ sudo apt list libvulkan1 mesa-vulkan-drivers glslc 
glslc/stable,now 2023.2-1 amd64 [installed]
libvulkan1/stable,now 1.3.239.0-1 amd64 [installed,automatic]
mesa-vulkan-drivers/stable,now 22.3.6-1+deb12u1 amd64 [installed]

In order to reproduce:

git checkout b4280
rm -rf build/
cmake -B build -DGGML_VULKAN=ON -DGGML_CCACHE=OFF
cmake --build build

The build properly compiles vulkan-shaders-gen, but then it hangs with multiple errors following this pattern (see full log below):

...
[  5%] Linking CXX executable ../../../../bin/vulkan-shaders-gen
[  5%] Built target vulkan-shaders-gen
[  6%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
cannot compile matmul_f32_f16_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_coopmat.spv -DACC_TYPE=float -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

The previous tag (b4279) builds fine with the same setup.

Is it possible to disable this extension?

If not, and if Vulkan is not supported any more on such platforms, it could be useful to have a version check in CMake, so that people can understand right away that this is a problem with the version of a dependency.

First Bad Commit

3df784b

Relevant log output

$ cmake -B build -DGGML_VULKAN=ON -DGGML_CCACHE=OFF
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.39.5") 
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Adding CPU backend variant ggml-cpu: -march=native 
-- Found Vulkan: /usr/lib/x86_64-linux-gnu/libvulkan.so (found version "1.3.239") found components: glslc glslangValidator 
-- Vulkan found
-- Including Vulkan backend
-- Configuring done
-- Generating done
-- Build files have been written to: xxx/llama.cpp/build

$ $ cmake --build build
...
[  4%] Linking CXX shared library libggml-base.so
[  4%] Built target ggml-base
[  4%] Building CXX object ggml/src/ggml-vulkan/vulkan-shaders/CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.cpp.o
[  5%] Linking CXX executable ../../../../bin/vulkan-shaders-gen
[  5%] Built target vulkan-shaders-gen
[  6%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
cannot compile matmul_f32_f16_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_coopmat.spv -DACC_TYPE=float -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_f32_f16_aligned_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=f16mat2x4 -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=8 -DLOAD_VEC_B=8 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_f16_aligned_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=f16mat2x4 -DCOOPMAT=1 -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=8 -DLOAD_VEC_B=8 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_f16_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_coopmat.spv -DACC_TYPE=float -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_f32_f32_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f32_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_f32_f32_aligned_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f32_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=mat2x4 -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=8 -DLOAD_VEC_B=8 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_f16_f32_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_f32_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DCOOPMAT=1 -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_q4_0_f32_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f32_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DCOOPMAT=1 -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_f16_f32_aligned_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_f32_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=mat2x4 -DCOOPMAT=1 -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=8 -DLOAD_VEC_B=8 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_q4_0_f32_aligned_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f32_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=mat2x4 -DCOOPMAT=1 -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 -DLOAD_VEC_B=8 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_q4_0_f16_aligned_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=f16mat2x4 -DCOOPMAT=1 -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 -DLOAD_VEC_B=8 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_q4_0_f16_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_q4_1_f32_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f32_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DCOOPMAT=1 -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_q4_1_f16_aligned_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=f16mat2x4 -DCOOPMAT=1 -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 -DLOAD_VEC_B=8 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_q4_1_f32_aligned_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f32_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=mat2x4 -DCOOPMAT=1 -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 -DLOAD_VEC_B=8 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_q4_1_f16_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

[0cc4m]

There is a check in CMake for coopmat support that should be preventing this: https://github.com/ggerganov/llama.cpp/blob/f66f5829276650cd83a087ab2cfed1a760183ea1/ggml/src/ggml-vulkan/CMakeLists.txt#L11-L23

But I don't see the output of that check in your log. Are you sure it was using the master branch?

[mbaudier]

Yes, I have seen the test you are refering to, but it was added slightly later, in b4287 (ecc93d0), while the logs I have provided were from the first failing build (b4280).

As I understand it, this tests for GL_NV_cooperative_matrix2, not GL_KHR_cooperative_matrix.

To be on the safe side, I have tried with the current state of master (e7da954) :

$ cmake -B build -DGGML_VULKAN=ON -DGGML_CCACHE=OFF
...
-- Vulkan found
-- GL_NV_cooperative_matrix2 not supported by glslc
-- Including Vulkan backend
...

$ cmake --build build
...
[  4%] Linking CXX executable ../../../../bin/vulkan-shaders-gen
[  4%] Built target vulkan-shaders-gen
[  5%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
cannot compile matmul_f32_f16_coopmat

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2  lama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_coopmat.spv -DACC_TYPE=float -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t 

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

and the same issue appears.

[0cc4m]

As I understand it, this tests for GL_NV_cooperative_matrix2, not GL_KHR_cooperative_matrix.

You are correct, I guess I wasn't awake enough yet...

Basically you need newer Vulkan headers and a newer glslc version. You could use a ppa or build them yourself. It's unfortunate that the versions Debian provides are that old.

Maybe that should also be checked in a similar way to coopmat2, but for now that has not been implemented.

[mbaudier]

Thank you for your quick answer! We can live without Vulkan support until the next Debian Stable.

[pepijndevos]

So are all debian bookworm users just expected to stay on b4279?

It seems an environment variable GGML_VK_DISABLE_COOPMAT is introduced, why not a build define?
This does seem to happen for GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT.

[0cc4m]

So are all debian bookworm users just expected to stay on b4279?

It seems an environment variable GGML_VK_DISABLE_COOPMAT is introduced, why not a build define? This does seem to happen for GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT.

As mentioned earlier, you can install glslc and Vulkan headers separately from your distro's versions. You can install the whole Vulkan SDK.

The environment variable was introduced because it's a single line of code, and required for performance comparison tests. A build define or a check similar to the one for coopmat2 would be fine, you're welcome to contribute it if you need it quickly.

[pepijndevos]

Maybe a better solution would be to add a newer vulkan to bookworm-backports, I might look into that.

[mbaudier]

Following @0cc4m's recommendation, I reproduced the GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT logic and disabled the parts related to coopmat in:

• vulkan-shaders-gen.cpp
• ggml-vulkan.cpp

until I could get it to build locally on my Debian 12/bookworm workstation:

$ cmake ../../llama.cpp -DGGML_VULKAN=ON
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.39.5") 
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- ccache found, compilation results will be cached. Disable with GGML_CCACHE=OFF.
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native 
-- Found Vulkan: /usr/lib/x86_64-linux-gnu/libvulkan.so (found version "1.3.239") found components: glslc glslangValidator 
-- Vulkan found
-- GL_NV_cooperative_matrix2 not supported by glslc
-- GL_KHR_cooperative_matrix not supported by glslc
-- Including Vulkan backend
-- Configuring done
-- Generating done

$ cmake --build .
...
[ 12%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o
[ 13%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan-shaders.cpp.o
/home/mbaudier/dev/git/unstable/tp/org.argeo.tp.ml/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘void ggml_vk_print_gpu_info(size_t)’:
/home/mbaudier/dev/git/unstable/tp/org.argeo.tp.ml/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2606:26: warning: unused variable ‘last_struct’ [-Wunused-variable]
 2606 |     VkBaseOutStructure * last_struct = (VkBaseOutStructure *)&vk12_features;
      |                          ^~~~~~~~~~~
[ 13%] Linking CXX shared library libggml-vulkan.so
[ 13%] Built target ggml-vulkan
...

I could then do a quick test:

$ bin/llama-cli --list-devices
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = NVIDIA RTX A3000 Laptop GPU (NVIDIA) | uma: 0 | fp16: 1 | warp size: 32 | matrix cores: none
Available devices:
  Vulkan0: NVIDIA RTX A3000 Laptop GPU (6144 MiB, 6144 MiB free)

$ bin/llama-cli --conversation --prompt "You are a helpful assistant" --model /srv/ml/models/hugging-quants/llama-3.2-3b-instruct-q4_k_m.gguf --device Vulkan0 --gpu-layers 29
...
ggml_vulkan: Compiling shaders...................................................Done!
llm_load_tensors: offloading 28 repeating layers to GPU
llm_load_tensors: offloading output layer to GPU
llm_load_tensors: offloaded 29/29 layers to GPU
llm_load_tensors:      Vulkan0 model buffer size =  1918.35 MiB
llm_load_tensors:   CPU_Mapped model buffer size =   308.23 MiB
...
You are a helpful assistant
> Who are you?
I'm an AI assistant, designed to help and support users with information and tasks.
...

The changes are available in this branch of my forked repo:
https://github.com/mbaudier/llama.cpp/tree/debian-bookworm-vulkan

This should definitely be reviewed carefully and tested on an environment where these Vulkan extensions are actually available. As already mentioned, I have no experience at all with Vulkan development, but I am used to hack and patch third-party software for packaging and compatibility purposes.

Do you think that it goes in the right direction?
How can I check whether my GPU would support these extensions with a newer Vulkan SDK?

This would definitely be great to be able to have standards-based GPU support on plain Debian Stable.

[0cc4m]

@mbaudier Yeah, that looks good. Just open a PR with the changes and I'll test it, I don't think more than minor changes would be needed to merge.

How can I check whether my GPU would support these extensions with a newer Vulkan SDK?

https://vulkan.gpuinfo.org/ is a good resource on Vulkan extension support. But in this case, it's very simple: Only Nvidia RTX and AMD RX 7000 GPUs support the coopmat extension. Intel ARC kinda does, but it's not mature yet.