BLD: Replace CPU-only builds with Vulkan on PyPI

[iwr-redmond]

Now that #61 and #89 have been completed, it may be helpful to retire CPU-only Windows and Linux builds in favor of the Vulkan ones when publishing to PyPI, as these newer builds also include the CPU optimizations noted by @codingl2k1 here.

This would compliment the existing OSX Metal wheels that are already published to PyPI, allowing ordinary users to access their NVIDIA, AMD, and Intel GPUs without needing to use extra indices. CUDA and ROCm builds would continue to be made available separately along with the legacy Vulkan on OSX Intel wheels.

[codingl2k1]

Good suggestion. However, the PyPI Linux wheel uses glibc 2.17, while the Vulkan Linux build requires glibc 2.35. This will downgrade compatibility.

[iwr-redmond]

Can you downgrade the Vulkan builds to glibc2.17?

[codingl2k1]

Can you downgrade the Vulkan builds to glibc2.17?

It requires manylinux, and GitHub has already dropped support for older Linux systems in GitHub Actions runners. Additionally, the Vulkan SDK itself may require a higher version of glibc. I can add a custom PyPI index for Vulkan builds, similar to CUDA: https://xorbitsai.github.io/xllamacpp/whl/cu124/

[iwr-redmond]

That would be a great start. Thanks!

[iwr-redmond]

the PyPI Linux wheel uses glibc 2.17, while the Vulkan Linux build requires glibc 2.35

Looking at this more closely with the cibuildwheel documentation, it looks like the CPU build specifies manylinux_x86_64, which defaults to glibc 2.28. According to Distrowatch, that gives compatibility back to Ubuntu LTS 20.04 and RHEL 8.10.

I reckon this means compatibility could be achieved by modifying the non-CPU workflow at L242:

- auditwheel repair --plat manylinux_2_35_x86_64 dist/*.whl -w dist
+ auditwheel repair --plat manylinux_2_28_x86_64 dist/*.whl -w dist

Is that correct?

[codingl2k1]

the PyPI Linux wheel uses glibc 2.17, while the Vulkan Linux build requires glibc 2.35

Looking at this more closely with the cibuildwheel documentation, it looks like the CPU build specifies manylinux_x86_64, which defaults to glibc 2.28. According to Distrowatch, that gives compatibility back to Ubuntu LTS 20.04 and RHEL 8.10.

I reckon this means compatibility could be achieved by modifying the non-CPU workflow at L242:

• auditwheel repair --plat manylinux_2_35_x86_64 dist/*.whl -w dist

• auditwheel repair --plat manylinux_2_28_x86_64 dist/*.whl -w dist
  Is that correct?

Sadly, we can't pin a lower version of glibc, as indicated by the logs from the last Vulkan Linux release build:

xllamacpp-0.2.4-cp39-cp39-linux_x86_64.whl is consistent with the
following platform tag: "linux_x86_64".

The wheel references external versioned symbols in these
system-provided shared libraries: libgomp.so.1 with versions
{'OMP_1.0', 'GOMP_4.0', 'GOMP_1.0'}, libm.so.6 with versions
{'GLIBC_2.27', 'GLIBC_2.29', 'GLIBC_2.2.5'}, libstdc++.so.6 with
versions {'GLIBCXX_3.4.17', 'GLIBCXX_3.4.26', 'GLIBCXX_3.4.25',
'CXXABI_1.3.2', 'GLIBCXX_3.4.20', 'CXXABI_1.3.3', 'CXXABI_1.3.13',
'CXXABI_1.3.9', 'GLIBCXX_3.4.21', 'GLIBCXX_3.4.22', 'GLIBCXX_3.4.18',
'GLIBCXX_3.4.30', 'GLIBCXX_3.4.15', 'GLIBCXX_3.4.14',
'GLIBCXX_3.4.29', 'GLIBCXX_3.4.11', 'GLIBCXX_3.4.19', 'CXXABI_1.3.11',
'CXXABI_1.3.5', 'GLIBCXX_3.4.9', 'GLIBCXX_3.4', 'CXXABI_1.3',
'CXXABI_1.3.7'}, libc.so.6 with versions {'GLIBC_2.32', 'GLIBC_2.4',
'GLIBC_2.14', 'GLIBC_2.29', 'GLIBC_2.7', 'GLIBC_2.34', 'GLIBC_2.3',
'GLIBC_2.6', 'GLIBC_2.17', 'GLIBC_2.33', 'GLIBC_2.3.4', 'GLIBC_2.2.5'}

This constrains the platform tag to "manylinux_2_35_x86_64". In order
to achieve a more compatible tag, you would need to recompile a new
wheel from source on a system with earlier versions of these
libraries, such as a recent manylinux image.

You can see This constrains the platform tag to "manylinux_2_35_x86_64".

[iwr-redmond]

Thanks for this. What I don't yet understand is how the CPU build is compiled with the lower glibc version. Is it a private runner?