Compile bug: Vulkan build fails on GL_KHR_cooperative_matrix

[theaerotoad]

Git commit

1a31d0d

Operating systems

Linux

GGML backends

Vulkan

Problem description & steps to reproduce

Trying to update llama.cpp build to test out the latest Vulkan improvements. I think that 10597 is the culprit for me, as 3df784b (tag: b4280) hangs compilation.

Reverting back to 86a1934 (commit right before) works, and but nothing after this compiles.

I'm building on Debian Linux, on a Framework AMD64. I'm having the same issue on another AMD machine as well.

GLSLC

glslc --version
shaderc 2023.2-1
spirv-tools 2022.4+1.3.236.0-1
glslang 11.13.0-1

Target: SPIR-V 1.0

libvulkan-dev

1.3.239.0-1

First Bad Commit

3df784b

Relevant log output

...
[ 20%] Linking CXX executable ../../../../bin/vulkan-shaders-gen                                                                                                                          
[ 20%] Built target vulkan-shaders-gen                                                       
[ 20%] Generate vulkan shaders                                                               
ggml_vulkan: Generating and compiling shaders to SPIR-V                                                                                                                                   cannot compile matmul_f32_f16_coopmat                                                                                                                                                                                                                                                                                                                                               
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2  /home/user/software/llama.cpp_latest/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/user/software/
llama.cpp_build.20241211/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_coopmat.spv -DACC_TYPE=float -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFL
OAT_TYPE=float16_t                                                                                                                                                                        
                                                                                                                                                                                          
/home/user/software/llama.cpp_latest/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
/home/user/software/llama.cpp_latest/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier                         
/home/user/software/llama.cpp_latest/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' :  syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.

cannot compile matmul_f16_aligned_coopmat
...

[0cc4m]

Your glslc is too old. You could build it from source: https://github.com/google/shaderc#getting-and-building-shaderc

Maybe we should support older versions, but that would require further work to test whether the extension is actually available.

[theaerotoad]

Yup--came across this here as well. Looks like I need at least 1.3.261.0 or newer. Thanks @0cc4m.

[jeffbolznv]

Should we consider switching to building shaderc from a known good commit? We have multiple known spirv-tools bugs, a couple features that are "recent" and not available on certain systems, and it's just generally concerning that we aren't all building the same code.

[theaerotoad]

Well, it'd be helpful for me, if there was detection of some base Vulkan/shader version, vs cutting edge. I pulled down a pre-built glslc, got further and other compile errors, then started wondering if I could run in a docker or qemu with Vulkan pass-through.

Vulkan appears to be the backend that is likely to support the widest range of devices, so there may be a case for ensuring you can build llama.cpp against a stable version that's widely available and stable.

[0cc4m]

That might be a good idea, but it would have to be implemented in a way that works on all systems.

[jeffbolznv]

@0cc4m I think you had tried it in your early development of the vulkan backend, and eventually abandoned it. Do you recall the issues?

[0cc4m]

@0cc4m I think you had tried it in your early development of the vulkan backend, and eventually abandoned it. Do you recall the issues?

I tried a bunch of things, not sure which one you mean.

At some point I tried building at runtime by linking to shaderc. It works, but there was trouble with Windows and of course it takes more time when starting the program. Not ideal.

Later I had a Python script that generated the SPIR-V. The compiled shaders were kept in git and the script only had to be run when you changed shaders. This was nice because it meant someone building the library didn't need glslc at all. But it also bloated the repo with a pretty big and kinda binary file.

The current approach is better, but has some caveats due to the different glslc versions. We can get around that with submodules and building shaderc as well, but that could also be an annoying increase in git complexity, cmake scripting and build time.

[theaerotoad]

A follow-up here, in case someone comes across this later. I ended up needing (I think) to install the whole Vulkan SDK, rather than just upgrading glslc.

Pulled the SDK from https://vulkan.lunarg.com/sdk/home#linux and am able to compile again.