Release LLVM versions 20 and 21

[alan-j-hu]

Also remove stray newline in some older LLVM opam files

[raphael-proust]

ty for the contribution

whenever possible, in order to make the ci results easier to parse, we try to separate cosmetic changes (such as insignificant newline removal) from significant changes, no worries though: i'll open a PR for the whitespace stuff, but just in case for next time :)

[raphael-proust]

reopened to retrigger the CI

[jmid]

The close-open-dance only retriggers the Windows workflows.
For opam-ci one has to press Rebuild all on the workflows page (I just did FYI)

[mseri]

In some of the failures there seems to be an issue with the version extraction, can you have a look?

[alan-j-hu]

@mseri My configure.sh script checks the presence of the static/shared libs with the following code

if ! $llvm_config --link-${mode} --libs; then
    echo "Note: '$llvm_config' matches version '$version', but does not support link mode '$mode'."
    continue
fi

I think the CI failures you are referring to are the ones which look like:

# + for llvm_config in llvm-config-$version llvm-config${version} llvm-config-${version}.0 llvm-config${version}0 llvm-config-mp-$version llvm-config-mp-${version}.0 llvm${version}-config llvm-config-${version}-32 llvm-config-${version}-64 llvm-config $brew_llvm_config
# ++ llvm-config-20 --version
# + llvm_version=20.1.8
# + case $llvm_version in
# + llvm-config-20 --link-static --libs
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMDemangle.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMSupport.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMTableGen.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMTargetParser.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMBinaryFormat.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMBitstreamReader.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMRemarks.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMCore.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMFuzzerCLI.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMBitReader.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMMC.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMAsmParser.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMIRReader.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMMCParser.a
# llvm-config: error: missing: /usr/lib/llvm20/lib/libLLVMTextAPI.a

To my understanding, this failure means that the distro does not install the static library archives. So, static LLVM is not supported. However, I don't know because I don't have access to those failing distros.

[mseri]

Sorry for the late reply. Indeed that was expected, my bad, but there are also "shared" failures, where at least in some cases the issue seem to be with libzstd. For example

• macos-homebrew-ocaml-4.14-arm64 (experimental)
  • llvm.20-shared (failed: linker command failed with exit code 1 (use -v to see invocation))
  • llvm.21-shared (failed: linker command failed with exit code 1 (use -v to see invocation))
• macos-homebrew-ocaml-5.4-arm64 (experimental)
  • llvm.20-shared (failed: linker command failed with exit code 1 (use -v to see invocation))
  • llvm.21-shared (failed: linker command failed with exit code 1 (use -v to see invocation))

Opensuse (for both ocaml 4.14 and 5.4, not all linked here):

• llvm.20-shared (failed: ld returned 1 exit status)
• llvm.21-shared (failed: ld returned 1 exit status)

Archlinux (also not all linked, maybe needs a conf python?)

• llvm.20-shared (failed: The compilation of llvm.20-shared failed at "bash -ex install.sh build --llvm-config llvm-config-20 --libdir /home/opam/.opam/5.4/lib --cmake cmake --make make --link-mode shared".)

And also freebsd

Can we do something about it?

[alan-j-hu]

Looks like OpenSUSE installs /usr/lib64/libLLVM.so, while the more common convention is to install libLLVM-version.so. See https://opensuse.pkgs.org/tumbleweed/opensuse-oss-x86_64/llvm21-devel-21.1.6-1.1.x86_64.rpm.html, and this patch file: https://build.opensuse.org/projects/openSUSE:Factory/packages/llvm21/files/llvm-normally-versioned-libllvm.patch?expand=1.

Is there a way for opam to selectively apply patches depending on the operating system? OpenSUSE will need a different patch from the default, which looks for LLVM-21: https://github.com/alan-j-hu/llvm-ocaml-ci/blob/3601bff13496ac980c7352b98c75da883327c220/llvm/AddOCaml.cmake.patch.19#L62

[mseri]

@rjbou is that possible with opam?

[alan-j-hu]

According to the opam manual, it is possible to selectively apply patches using filters: https://opam.ocaml.org/doc/Manual.html#opamfield-patches.

I am currently trying to fix some of these build issues. However, I recently started a new job that I'm getting acclimated to, and I have very limited time in the day to work on this.

[alan-j-hu]

FYI, I'm feeling very frustrated and anxious right now. My unfamiliarity with the operating systems at hand is causing me to make all sorts of mistakes, and I have very limited free time to work on this. I might need to take a step back, and I don't know if I'll be able to solve these issues right now.

[mseri]

Since this looks to be quite complex, my suggestion would be to merge the current version as is, not blocking it on opensuse. So that if users need the package and find the error, can figure out the best way to deal with that and contribute to the script. What do you think?

[jmid]

@alan-j-hu - take your time. Starting a new job is indeed stressful.

As for this PR, I suggest tackling some of the lower-hanging red CI lights before merging. According to https://pkgs.org/search/?q=libLLVM.so the Centos package names are slightly off.

[jmid]

OK, I was too fast above, so I reverted my commits - and attempted x-ci-accept-failures silencing,
only to be reminded that these don't apply to system packages... 😬 🤷

Now, there are Centos AppStream packages for llvm-devel {20,21} as witnessed by
https://pkgs.org/search/?q=libLLVM.so which I've double checked here:
https://mirror.stream.centos.org/9-stream/AppStream/x86_64/os/Packages/

How do we install specific major versions {20,21} without locking in to, e.g., llvm-devel-20.1.8-3? 🤔

This is a very welcome PR, but I see no rush to merging it. I still think we should address some of the many Alpine, Arch, Centos, Fedora, FreeBSD, and macOS CI red lights (as pointed out by @mseri above) and not get too focused on OpenSuse alone.

[alan-j-hu]

Thank you for your patience, everyone. I've pushed some new code that hopefully should fix the macos and opensuse build errors.

I think what happened with macos is that I was relying on the environment variable HOMEBREW_PREFIX, but this variable wasn't always defined for some reason. When I tested with Linuxbrew, it was defined, but when I tested on a Mac, it was not. Therefore, I retrieve the prefix using brew --prefix instead.

Based on what I found on https://build.opensuse.org/projects/openSUSE:Factory/packages/llvm21/files/llvm-normally-versioned-libllvm.patch?expand=1, I think that OpenSUSE patches its LLVM package to not have the major version suffixed to the name. Therefore, I added a patch file to remove that only for the OpenSUSE build, which I hope will fix the issue. However, I don't have access to an OpenSUSE installation to actually test the build on.

If these changes fix the aforementioned issues, I think this PR should be merged so that other people can use the LLVM 20 and 21 bindings, and possibly contribute fixes to build issues.

[alan-j-hu]

Made an adjustment: When linking to the LLVM shared library, you can build native, bytecode, and "custom runtime" bytecode executables (.exe, .bc, and .exe in Dune). However, when linking to the LLVM static library, you cannot build normal bytecode (.bc) executables, only "custom runtime" (.bc.exe) executables. I made the change so that the -custom flag is only passed in when using the static version of these bindings.