Run __init__() even in sysimage builds

[topolarity]

This change makes the LinearAlgebra initialization code run during the sysimage build, similar to how it would run when using LinearAlgebra at top-level in a normal package. In this configuration, we can't rely on the serializer to isolate side-effects for us, so we have to manually ensure that any mutations that we perform here are not accidentally persisted to run-time.

Requires JuliaLang/julia#59232 and JuliaLang/julia#59233

Resolves the segfault in JuliaLang/julia#59215

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.86%. Comparing base (2c3fe9b) to head (e365477).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1407      +/-   ##
==========================================
- Coverage   93.87%   93.86%   -0.02%     
==========================================
  Files          34       34              
  Lines       15830    15831       +1     
==========================================
- Hits        14861    14859       -2     
- Misses        969      972       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[KristofferC]

Runningg any non-trivial code in the sysimage build process seems like it should almost not be done since that Julia state is so dysfunctional. In eg PackageCompiler we compile packages by themselves in the normal way before they are loaded into he sysimage (so that the top level code run in a normal julia environment).

[topolarity]

Yeah, I tend to agree.

I do think this approach is sound, but it's much simpler just to ensure that all of our pipelines respect the same compilation phasing. Changing juliac to use pkgimages for all top-level execution is definitely a better fix.

Closing in preference of that

[JeffBezanson]

That's fine but it feels really broken that we have a mode where you can load blas and call it but it segfaults. Why not somehow let it initialize its entry points every time it's loaded? In an alternate universe, every blas call uses a OncePerProcess to initialize the library (note that may not actually be sufficient, but just supposing) and everything works. If we had done that, what would be the problem it caused?

[KristofferC]

All I am saying is that the way the output process is set up (to not run inits) makes running code in it brittle and potentially dangerous. For example, at some point, rand() always returned 0.0 because the seeding in init was not done (see the https://github.com/JuliaLang/PackageCompiler.jl/blob/57505caad062b5b1985bcbbb2fb15b9b6e9e6d99/examples/MyApp/src/MyApp.jl#L71-L72 test).

[topolarity]

Yeah, there's nothing fundamentally stopping us from executing things correctly during the sysimage build, it's true.

We just kind of have to go all the way down the dependency tree and make sure that any __init__s we need are idempotent + run eagerly - in principle, that would make all functionality available early (rand() is just another example of an __init__ dependency that needs to run eagerly after we verify that it is idempotent)

But if we are running significant code in the sysimage build, we also have to worry about making sure any side-effects are ephemeral (e.g. JuliaLang/julia#59232), which a lot of the internal hooks / registration functions in Base aren't quite ready for. It's a fairly long list of things (anything that touches global mutable state or an __init__ in Base) that would need updating to be ephemeral / OncePerProcess as expected.

And there's some awkward stuff too where JLL's have semi-public globals that we probably can't directly replace with OncePerProcess, for breakage reasons, so they have to keep around build-time garbage that gets replaced by the runtime __init__ if we run these eagerly.

I'm not against starting that process (or to taking targeted fixes specifically for 1.12), but I think it may not be very important to us if the plan for 1.13 is to restore the "canonical" __init__ behavior for juliac which was the main instigator of this bad "sysimage code execution"