Compressed sysimage using zstd (manual 1.11 backport) #59340
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
That looks unrelated (was this the magic github issue number autocompletion?) |
|
The PR #59227 should be linked instead, I think. |
topolarity
changed the base branch from
backports-release-1.11
to
release-1.11
topolarity
force-pushed
the
compressed-sysimg-1.11
branch
from
1fe2358 to
82e2dc4
Compare
Inspired by a bug I hit on MinGW recently with weak symbol resolution. I'm not sure why we started using these in 70000ac, since I believe we should be able to lookup these symbols just fine in our `jl_exe_handle` as long as it was linked / compiled with `-rdynamic` Migrating away from weak symbols seems a good idea, given their uneven implementation across platforms. (cherry picked from commit 4a6ada6)
Introduce `jl_image_buf_t` to represent the in-memory details of an unparsed system image and clean-up several global variables and split loading logic in staticdata.c so that we get (almost) everything we need from the sysimage handle at once. This allows sysimage loading to be separated into three phases: 1. Lookup the raw sysimage buffer as a `jl_image_buf_t` 2. For .so sysimages, parse the sysimage and initialize JIT targets, etc. 3. Finally load the sysimage into a `jl_image_t` Care was taken to preserve the existing behavior of calling `jl_set_sysimg_so` to configure the sysimage before initializing Julia, although this is likely to be next on the chopping block in a follow-up. Dependent on JuliaLang#57523.
Co-authored-by: Gabriel Baraldi <[email protected]> madvise with MADV_WILLNEED before accessing sysimage pages MADV_WILLNEED sysimage pages before checksumming or decompressing Even without compression, this gives about an 8% improvement in load times. Fix using jl_page_size before it is initialized Add zstd to ANALYSIS_DEPS now that it is used in staticdata.c Fix jl_page_size being uninitialized in jl_prefetch_system_image Add --compress-sysimage flag; use zstd-15 when enabled
topolarity
force-pushed
the
compressed-sysimg-1.11
branch
from
82e2dc4 to
5c227a2
Compare
|
Looks like the Took the liberty of re-targeting this to |
xal-0
changed the title
This is what I get for trusting the GitHub textbox instead of pasting in text |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is #59227 manually backported to 1.11, along with the dependencies #57523,
#57542, and a modified version of #58344 that adds the zstd jll but does not
make any of the LLVM changes.