[LLJIT] Symbols not found: [ llvm_orc_registerEHFrameSectionWrapper ]

[g0blue]

After updating to LLVM 17 we started seeing the error in title when creating an LLJIT instance when running on Linux. After digging through the source code and reading issue #61712 I think I understand the issue.

Why must functions like llvm_orc_registerEHFrameSectionWrapper be exported and found via dlsym? This seems like a bad design decision.

Our use case is perhaps unique, but we must support multiple versions of LLVM being used in the same process simultaneously. We have one shared library using LLVM to generate executable code for a future HW product, and another shared library using LLVM (of a potentially different version) to translate that binary (using LLJIT) into something that can be executed by current HW. The second shared library (using LLJIT) is linked with -Wl,--exclude,ALL to prevent export of symbols from static libraries. Additionally, we load this library using dlopen with the RTLD_LOCAL flag. Either of these will break LLJIT on LLVM 17.

All of this worked fine for the last 7-8 years until LLVM 17.

Is there any kind of workaround we can do (in client code only) to prevent the use of dlsym when creating an LLJIT instance?

[g0blue]

For anyone else dealing with this, I did find a workaround.

I had to remove -Wl,--exclude,ALL from the link parameters of my shared library that embeds LLVM, and I had to load the library with the RTLD_GLOBAL flag. That allowed LLVM to function correctly. However, it also broke the encapsulation of many components (not just LLVM) and caused many of our use cases to fail. The fix for that was some cmake magic (inspired by export_executable_symbols within AddLLVM.cmake) that iterated all static library dependencies of the shared library and excluded them one by one, with the exception of the LLVMOrcTargetProcess library. This cmake change removed about ~19000 unwanted exports (not all LLVM) and left us with just 9 LLVM related exports, including the 2 that are required for proper LLJIT functionality.

I still believe that requiring any LLVM exports breaks encapsulation and is very undesirable.

[lriggs]

Just wanted to add a vote to this issue. This problem is causing issues in Apache Arrow and is preventing me from upgrading our software which relies on Arrow.

This has been open for a year now, are there any plans to fix it?