Build Enzyme on mingw in CI

[ZuseZ4]

Status quo: We are able to build Enzyme on macos and linux in CI, and libload (dlopen) it into rustc.
@kiranshila tried to help me with getting Windows (MinGW) builds up. MSVC ins't supported due to some black magic in Enzyme.

On Linux, we have rustc linking dynamically against LLVM, so libEnzyme also dynamically links against it.
On Apple, we use static linking, so at first libEnzyme also tried to statically link LLVM, and we ended up with two LLVM copies, which causes bugs. We solved this via cmake flags, telling libEnzyme to look for llvm symbols only at dlopen time. This way libEnzyme reuses the LLVM symbols from rustc.

On MinGW, we also use static linking, and due to the limited number of public symbols per library, we can't really change that from my understanding. Unlike Macos, we can not tell the linker to resolve LLVM symbols at dlopen time (the flags used for MacOS aren't recognized), but we also can't ship a second LLVM copy. So, from our limited understanding of linkers, there isn't really a way around statically linking Enzyme into rustc and just always distributing it?

I got permission from the infra team to do that a while ago, even though it is not the preferred solution. We also had static linking of rustc and enzyme work in the past, so it should be doable to get it to work again. I just hate that we have 3 different approaches on 3 systems, but I guess that's just the way it is.

@bjorn3 does that sound sensible to you, or do you see a better solution for mingw?

fyi @Kobzol

I don't necessarily intend to block nightly release on MinGW, but since we're waiting on #150992 anyway, I thought it would be nice to try to get Windows in as well.

Summary

Command used

https://rustc-dev-guide.rust-lang.org/autodiff/installation.html#build-instructions

Expected behaviour

Actual behaviour

A variety of errors based on the concrete setup. When trying to copy the MacOS solution, Kiran ended up with:

undefined reference to `llvm::llvm_unreachable_internal(char const*, char const*, unsigned int)'****

Bootstrap configuration (bootstrap.toml)

<config>

Operating system

HEAD

Additional context

Build Log

<log>

[ZuseZ4]

Also, if we follow Nikitas suggestion from #151243 (comment), we would at least share the same setup for Linux and Apple, which would be nice. I don't want to block nightly support on even more changes though, so I'll keep those two as they are (for now).

[mati865]

There is one Windows configuration that supports dynamic LLVM library if you are interested.

I'll let myself copy paste the details from davidlattimore/wild#1411 (comment):

When it comes to Windows, dynamic linking of libLLVM is supported only when building with Clang in MinGW mode: llvm/llvm-project@c5b3de6#diff-c32cf5f1ab46b94ceac7f7597aace44624436f439c8304c029feca005c000a4bR245-R247
Linked feature translates ELF symbol visibility into Windows directives that exclude symbols from being exported. That allows Clang to avoid that dumb PE ~2^16 exported symbols limit.

Enabling shared libllvm for windows-gnullvm should be as simple as adding the right argument to the buils invocation on CI.
For regular windows-gnu hosts that will be more complicated and comes with a few a catches.

[ZuseZ4]

I would be!
However, the reason Enzyme doesn't support msvc is that Enzyme overwrites a private function in LLVM, and due to said Limit MSVC is the only major compiler which checks for this. https://github.com/EnzymeAD/Enzyme/blob/12a5ee0d8dadeb7cafd60ec7a5e250d8287d566d/enzyme/Enzyme/CacheUtility.h#L32
I could imagine overwriting the visibility might also conflict with this clang mode?

[mati865]

If this works with Clang on Linux, I guess it'd be fine on Windows as well. It doesn't seem to affect symbol visibility attributes.