Creating system image hangs forever with multiple threads

[PatrickHaecker]

We have two issues in PackageCompiler.jl (PC#963 and PC#990) where PackageCompiler hangs forever when creating the final system image with multiple threads. The issues can be "fixed" by using only a single thread (PC#1042).

However, we already have a git bisect in these issues pointing to this commit indicating, that we might have a uv mutex multi-threading problem in Julia itself.
We also have something like a test case, because we know that it is triggered when the Cmd defined in this line is run e.g. with --threads=auto. We could, e.g., copy the involved files somewhere after printing the command and then call the new process manually to debug it.

So there seems to be a chance to not only plaster over the bug, but fix it for good if someone knowledgeable about Julia's multi-threading details (that excludes me) hunts it down.
As you, @vtjnash, seem to know the code as you probably made the commit which at least made the underlying problem observable, might I ask you to have a look?

[gbaraldi]

Precompilation is currently poorly defined when running with threads.

[PatrickHaecker]

Ok, thanks.
In 1.10 this problem was at least not observable. What are the options?

• Define how we want to have it and implement it accordingly.
• Make the critical section single-threaded (by a lock?). I assume the precompilation itself would still use multiple processes, so that the performance regression would be low.