Compile bug: I am trying to compile llama.cpp with Vulkan backend.

[acbits]

Git commit

$git rev-parse HEAD
d8359f5

Operating systems

Linux

GGML backends

Vulkan

Problem description & steps to reproduce

I am trying to compile llama.cpp with Vulkan backend. I have been building successfully with the same configuration for the past 1 year, but not version 6615.

First Bad Commit

Not sure as I haven't built since version 6431.

Compile command

export CFLAGS='-mtune=native -march=native -O3  -fno-omit-frame-pointer'
export CXXFLAGS=$CFLAGS

CXX=g++-13.4 CC=gcc-13.4 cmake -DCMAKE_INSTALL_PREFIX=/usr/local \
             -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=mold" \
             -DCMAKE_MODULE_LINKER_FLAGS="-fuse-ld=mold" \
             -DCMAKE_SHARED_LINKER_FLAGS="-fuse-ld=mold" \
             -DLLAMA_CURL=OFF \
             -DGGML_CCACHE=OFF  \
             -DGGML_VULKAN:BOOL=ON \
             -DBUILD_SHARED_LIBS:BOOL=OFF \
             -DCMAKE_BUILD_TYPE=Release ..

Relevant log output

Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In member function 'vk::PhysicalDeviceProperties vk::PhysicalDevice::getProperties(const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:243:62: error: could not convert 'ggml_vk_default_dispatcher()' from 'vk::detail::DispatchLoaderDynamic' to 'const vk::DispatchLoaderDynamic&'
  243 |     vk::PhysicalDeviceProperties props = device.getProperties();
      |                                          ~~~~~~~~~~~~~~~~~~~~^~
      |                                                              |
      |                                                              vk::detail::DispatchLoaderDynamic
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:243:62: note:   when instantiating default argument for call to 'vk::PhysicalDeviceProperties vk::PhysicalDevice::getProperties(const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function 'vk_device_architecture get_device_architecture(const vk::PhysicalDevice&)':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:243:62: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
In file included from /usr/include/vulkan/vulkan.hpp:8582,
                 from /media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:16:
/usr/include/vulkan/vulkan_funcs.hpp:284:90: note: in passing argument 1 of 'vk::PhysicalDeviceProperties vk::PhysicalDevice::getProperties(const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
  284 |                                          PhysicalDevice::getProperties( Dispatch const & d ) const VULKAN_HPP_NOEXCEPT
      |                                                                         ~~~~~~~~~~~~~~~~~^
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In member function 'typename vk::ResultValueType<std::vector<vk::ExtensionProperties, ExtensionPropertiesAllocator> >::type vk::PhysicalDevice::enumerateDeviceExtensionProperties(vk::Optional<const std::__cxx11::basic_string<char> >, const Dispatch&) const [with ExtensionPropertiesAllocator = std::allocator<vk::ExtensionProperties>; Dispatch = vk::DispatchLoaderDynamic; typename vk::ResultValueType<std::vector<vk::ExtensionProperties, ExtensionPropertiesAllocator> >::type = std::vector<vk::ExtensionProperties>]':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:246:105: error: could not convert 'ggml_vk_default_dispatcher()' from 'vk::detail::DispatchLoaderDynamic' to 'const vk::DispatchLoaderDynamic&'
  246 |         const std::vector<vk::ExtensionProperties> ext_props = device.enumerateDeviceExtensionProperties();
      |                                                                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
      |                                                                                                         |
      |                                                                                                         vk::detail::DispatchLoaderDynamic
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:246:105: note:   when instantiating default argument for call to 'typename vk::ResultValueType<std::vector<vk::ExtensionProperties, ExtensionPropertiesAllocator> >::type vk::PhysicalDevice::enumerateDeviceExtensionProperties(vk::Optional<const std::__cxx11::basic_string<char> >, const Dispatch&) const [with ExtensionPropertiesAllocator = std::allocator<vk::ExtensionProperties>; Dispatch = vk::DispatchLoaderDynamic; typename vk::ResultValueType<std::vector<vk::ExtensionProperties, ExtensionPropertiesAllocator> >::type = std::vector<vk::ExtensionProperties>]'
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function 'vk_device_architecture get_device_architecture(const vk::PhysicalDevice&)':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:246:105: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
/usr/include/vulkan/vulkan_funcs.hpp:607:132: note: in passing argument 2 of 'typename vk::ResultValueType<std::vector<vk::ExtensionProperties, ExtensionPropertiesAllocator> >::type vk::PhysicalDevice::enumerateDeviceExtensionProperties(vk::Optional<const std::__cxx11::basic_string<char> >, const Dispatch&) const [with ExtensionPropertiesAllocator = std::allocator<vk::ExtensionProperties>; Dispatch = vk::DispatchLoaderDynamic; typename vk::ResultValueType<std::vector<vk::ExtensionProperties, ExtensionPropertiesAllocator> >::type = std::vector<vk::ExtensionProperties>]'
  607 |             PhysicalDevice::enumerateDeviceExtensionProperties( Optional<const std::string> layerName, Dispatch const & d ) const
      |                                                                                                        ~~~~~~~~~~~~~~~~~^

/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In member function 'void vk::PhysicalDevice::getProperties2(vk::PhysicalDeviceProperties2*, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:275:30: error: could not convert 'ggml_vk_default_dispatcher()' from 'vk::detail::DispatchLoaderDynamic' to 'const vk::DispatchLoaderDynamic&'
  275 |         device.getProperties2(&props2);
      |         ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~
      |                              |
      |                              vk::detail::DispatchLoaderDynamic
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:275:30: note:   when instantiating default argument for call to 'void vk::PhysicalDevice::getProperties2(vk::PhysicalDeviceProperties2*, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function 'vk_device_architecture get_device_architecture(const vk::PhysicalDevice&)':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:275:30: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
/usr/include/vulkan/vulkan_funcs.hpp:6148:108: note: in passing argument 2 of 'void vk::PhysicalDevice::getProperties2(vk::PhysicalDeviceProperties2*, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
 6148 |                                                          Dispatch const &                                  d ) const VULKAN_HPP_NOEXCEPT
      |                                                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:291:105: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
  291 |         const std::vector<vk::ExtensionProperties> ext_props = device.enumerateDeviceExtensionProperties();
      |                                                                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/usr/include/vulkan/vulkan_funcs.hpp:607:132: note: in passing argument 2 of 'typename vk::ResultValueType<std::vector<vk::ExtensionProperties, ExtensionPropertiesAllocator> >::type vk::PhysicalDevice::enumerateDeviceExtensionProperties(vk::Optional<const std::__cxx11::basic_string<char> >, const Dispatch&) const [with ExtensionPropertiesAllocator = std::allocator<vk::ExtensionProperties>; Dispatch = vk::DispatchLoaderDynamic; typename vk::ResultValueType<std::vector<vk::ExtensionProperties, ExtensionPropertiesAllocator> >::type = std::vector<vk::ExtensionProperties>]'
  607 |             PhysicalDevice::enumerateDeviceExtensionProperties( Optional<const std::string> layerName, Dispatch const & d ) const
      |                                                                                                        ~~~~~~~~~~~~~~~~~^

/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:309:30: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
  309 |         device.getProperties2(&props2);
      |         ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~
/usr/include/vulkan/vulkan_funcs.hpp:6148:108: note: in passing argument 2 of 'void vk::PhysicalDevice::getProperties2(vk::PhysicalDeviceProperties2*, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
 6148 |                                                          Dispatch const &                                  d ) const VULKAN_HPP_NOEXCEPT
      |                                                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:319:105: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
  319 |         const std::vector<vk::ExtensionProperties> ext_props = device.enumerateDeviceExtensionProperties();
      |                                                                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/usr/include/vulkan/vulkan_funcs.hpp:607:132: note: in passing argument 2 of 'typename vk::ResultValueType<std::vector<vk::ExtensionProperties, ExtensionPropertiesAllocator> >::type vk::PhysicalDevice::enumerateDeviceExtensionProperties(vk::Optional<const std::__cxx11::basic_string<char> >, const Dispatch&) const [with ExtensionPropertiesAllocator = std::allocator<vk::ExtensionProperties>; Dispatch = vk::DispatchLoaderDynamic; typename vk::ResultValueType<std::vector<vk::ExtensionProperties, ExtensionPropertiesAllocator> >::type = std::vector<vk::ExtensionProperties>]'
  607 |             PhysicalDevice::enumerateDeviceExtensionProperties( Optional<const std::string> layerName, Dispatch const & d ) const
      |                                                                                                        ~~~~~~~~~~~~~~~~~^

/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In member function 'void vk::Device::destroyFence(vk::Fence, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:617:28: error: could not convert 'ggml_vk_default_dispatcher()' from 'vk::detail::DispatchLoaderDynamic' to 'const vk::DispatchLoaderDynamic&'
  617 |         device.destroyFence(fence);
      |         ~~~~~~~~~~~~~~~~~~~^~~~~~~
      |                            |
      |                            vk::detail::DispatchLoaderDynamic
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:617:28: note:   when instantiating default argument for call to 'void vk::Device::destroyFence(vk::Fence, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In destructor 'vk_device_struct::~vk_device_struct()':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:617:28: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
/usr/include/vulkan/vulkan_funcs.hpp:1569:106: note: in passing argument 3 of 'void vk::Device::destroyFence(vk::Fence, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
 1569 |                                                Dispatch const &                                          d ) const VULKAN_HPP_NOEXCEPT
      |                                                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In member function 'void vk::Device::destroyDescriptorSetLayout(vk::DescriptorSetLayout, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:634:42: error: could not convert 'ggml_vk_default_dispatcher()' from 'vk::detail::DispatchLoaderDynamic' to 'const vk::DispatchLoaderDynamic&'
  634 |         device.destroyDescriptorSetLayout(dsl);
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~
      |                                          |
      |                                          vk::detail::DispatchLoaderDynamic
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:634:42: note:   when instantiating default argument for call to 'void vk::Device::destroyDescriptorSetLayout(vk::DescriptorSetLayout, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In destructor 'vk_device_struct::~vk_device_struct()':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:634:42: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
/usr/include/vulkan/vulkan_funcs.hpp:3761:120: note: in passing argument 3 of 'void vk::Device::destroyDescriptorSetLayout(vk::DescriptorSetLayout, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
 3761 |                                                             Dispatch const &                                          d ) const VULKAN_HPP_NOEXCEPT
      |                                                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In member function 'void vk::Device::destroy(vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:636:23: error: could not convert 'ggml_vk_default_dispatcher()' from 'vk::detail::DispatchLoaderDynamic' to 'const vk::DispatchLoaderDynamic&'
  636 |         device.destroy();
      |         ~~~~~~~~~~~~~~^~
      |                       |
      |                       vk::detail::DispatchLoaderDynamic
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:636:23: note:   when instantiating default argument for call to 'void vk::Device::destroy(vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In destructor 'vk_device_struct::~vk_device_struct()':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:636:23: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
/usr/include/vulkan/vulkan_funcs.hpp:499:129: note: in passing argument 2 of 'void vk::Device::destroy(vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
  499 | PP_INLINE void Device::destroy( Optional<const VULKAN_HPP_NAMESPACE::AllocationCallbacks> allocator, Dispatch const & d ) const VULKAN_HPP_NOEXCEPT
      |                                                                                                      ~~~~~~~~~~~~~~~~~^

/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In member function 'vk::ResultValueType<vk::CommandPool>::type vk::Device::createCommandPool(const vk::CommandPoolCreateInfo&, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic; vk::ResultValueType<vk::CommandPool>::type = vk::CommandPool]':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:645:44: error: could not convert 'ggml_vk_default_dispatcher()' from 'vk::detail::DispatchLoaderDynamic' to 'const vk::DispatchLoaderDynamic&'
  645 |     pool = device->device.createCommandPool(command_pool_create_info);
      |            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                            |
      |                                            vk::detail::DispatchLoaderDynamic
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:645:44: note:   when instantiating default argument for call to 'vk::ResultValueType<vk::CommandPool>::type vk::Device::createCommandPool(const vk::CommandPoolCreateInfo&, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic; vk::ResultValueType<vk::CommandPool>::type = vk::CommandPool]'
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In member function 'void vk_command_pool::init(vk_device&, vk_queue*)':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:645:44: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
/usr/include/vulkan/vulkan_funcs.hpp:4409:90: note: in passing argument 3 of 'vk::ResultValueType<vk::CommandPool>::type vk::Device::createCommandPool(const vk::CommandPoolCreateInfo&, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic; vk::ResultValueType<vk::CommandPool>::type = vk::CommandPool]'
 4409 |                                Dispatch const &                                          d ) const
      |                                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In member function 'void vk::Device::destroyCommandPool(vk::CommandPool, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:649:30: error: could not convert 'ggml_vk_default_dispatcher()' from 'vk::detail::DispatchLoaderDynamic' to 'const vk::DispatchLoaderDynamic&'
  649 |     device.destroyCommandPool(pool);
      |     ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
      |                              |
      |                              vk::detail::DispatchLoaderDynamic
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:649:30: note:   when instantiating default argument for call to 'void vk::Device::destroyCommandPool(vk::CommandPool, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In member function 'void vk_command_pool::destroy(vk::Device&)':
/media/build/llama/ggml/src/ggml-vulkan/ggml-vulkan.cpp:649:30: error: invalid initialization of reference of type 'const vk::DispatchLoaderDynamic&' from expression of type 'vk::detail::DispatchLoaderDynamic'
/usr/include/vulkan/vulkan_funcs.hpp:4466:112: note: in passing argument 3 of 'void vk::Device::destroyCommandPool(vk::CommandPool, vk::Optional<const vk::AllocationCallbacks>, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]'
 4466 |                                                      Dispatch const &                                          d ) const VULKAN_HPP_NOEXCEPT
      |                                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^


The rest has been trimmed as it is too long.

[acbits]

OK. I was able to resolve the issue by upgrading vulkan headers to 1.3.302.

I thought cmake rules checked for the bare minimum version required for dependencies?

[lucifiel]

Build is broken with libvulkan-dev/noble,now 1.3.275.0-1build1 amd64 [installed] which is latest on Ubuntu 24.04.3 LTS

[lucifiel]

#16345

[geerlingguy]

Another 'me, too' for Raspberry Pi OS (based on Debian 12)'s defult Vulkan version.

[0cc4m]

Please check if this is fixed and close the issue if it is.

[acbits]

Yeah, this can be closed since others have verified that it works though I personally haven't tested it since it requires me to downgrade Vulkan headers.

[geerlingguy]

I can confirm compilation works with the latest commit again, without any changes.