RFC: Recommend and enable using Wild rather than Mold on Linux for local builds #37717
Conversation
cla-bot
bot
added
the
cla-signed
|
I think this needs some benchmarks of the output, wild doesn't support lto edit: or not..? davidlattimore/wild#1
no clue how this works! if anyone does I'd be interested to hear, otherwise disregard this i suppose. |
That's a great point, we should at least check it. Are there any scripted benchmarks that I can run? I don't see any in the tree. |
|
First off thanks for working on build speed! Every bit there helps. Though we might be jumping the gun on using Wild. They mention its not yet ready for production builds. We could mitigate that risk by limiting wild to debug builds. Then we get the speedup during development without the risk to releases. Do please test the impact on incremental builds, we cant have regressions there. |
sourcefrog
changed the title
Thanks for the feedback and encouragement. I had a look at how to turn it on only for Linux debug builds, but I can't see (at the moment) how to do it in
(Maybe there's some way around it.) I guess build scripts can both look at the platform and whether it's a debug build, but I think that would have to be done in every single package's Some options:
I'll also try to measure the performance on incremental builds and the runtime performance on tests. |
Damn.. I really thought that was possible. Thanks for investigating and clearly explaining why it its not possible atm.
Agreed
I agree with suggesting developers configure wild. This https://github.com/zed-industries/zed/blob/main/docs/src/development/linux.md#building-zed-for-linux might be a good place for a note. I'm not sure if it a good idea to set it in CI if we can guarantee everyone is running wild locally too. Debugging issues that only happen in CI caused by a linker that's different (that you are unaware of) seems hellish. If you had to manually setup wild locally you have a chance of recollecting that. Like you say it might be best to keep an eye on wild and once the warning not to use it in production get dropped put it in as default? |
|
OK so it sounds like a path forward is to
- [x] Measure incremental build and test execution performance with Wild vs Mold
- [ ] Suggest in the docs that developers use either Wild or Mold on Linux
- [x] Remove the in-tree override, so that user-level preferences for Wild aren't overridden
- [x] Use Mold in CI without relying on the .cargo/config.toml being checked in
Martin
|
Good plan! Lets go :) |
|
I realized the current in-tree config is a bit confusing if people also have Wild configured in their home directory, because cargo merges array values from multiple config file and so you end up with flags asking for both Mold and Wild, and presumably relying on one of them having priority inside rustc. So that's arguably a good reason to leave this to the user config, as some fraction of Rust devs may have Wild configured globally? Benchmarks
In all these benchmarks I'm ignoring the results from the first run to let it build and warm up, and then averaged 3 attempts.
Is there an automated benchmark for runtime performance? I didn't see one in the tree. This is on an AMD 5900X now with 64GiB, so a somewhat old machine. test suite performanceWild: 138s doctest performancedoctest builds and runs lots of small test targets, so arguably is a good test for this sort of thing. After an initial build of the tree, running Wild: 61s incremental rebuildIt looks like in jj worktrees the build stamp in Wild: 4.1s Smoke testI built 99b7167 with ConclusionI don't see any performance downside to recommending Wild on Linux. There is a complexity cost from people configuring it individually, and some risk that there could be bugs due to differences in behavior vs Mold. |
We have some benchmarks for hot paths/parts in zed (like the text representation (crates/rope/benches/). Though if it seems good enough perf wise to debug with for you its good enough to recommend.
Thats a 20% reduction! Amazing!
I fully agree with you there! Lets recommend the switch. If your looking for more build speedups btw bevy has a great compile performance section: https://bevy.org/learn/quick-start/getting-started/setup/#enable-fast-compiles-optional. |
|
I ran Rope benchmarks
With Wild Mold: |
This retains Mold in CI and lets developers choose anything locally.
sourcefrog
force-pushed
the
wild
branch
3 times, most recently
from
4625474 to
9471804
Compare
|
There's no significant detail in binary size: Wild's output is very slightly smaller. With Mold: With Wild: |
sourcefrog
changed the title
|
OK I think this is ready for another look. |
sourcefrog
force-pushed
the
wild
branch
2 times, most recently
from
46ed8de to
7c8fc00
Compare
|
Since Rust just changed the default on Linux to |
script/linux
Outdated
| fi | ||
| echo "Note: It's recommended to install and configure Wild or Mold for faster builds." | ||
| echo " Run script/install-wild or script/install-mold." | ||
| echo " Then see doc/src/development/linux.md to set your cargo config." |
There was a problem hiding this comment.
Could we link/be a bit more specific about where to look in linux.md?
Alternatively (if you feel comfortable with it) we can do the cargo config change from the script. (lets in that case also echo what we are doing and how to undo it)
There was a problem hiding this comment.
(but honestly at this point I am grasping at straws to give feedback, this all looks great!)
There was a problem hiding this comment.
I think changing the user wide config feels a bit intrusive for users who work on projects other than Zed.
Hopefully at some point we could decide it's reliable enough and go back to an in-tree config.
There was a problem hiding this comment.
I think changing the user wide config feels a bit intrusive for users who work on projects other than Zed.
Good point. Can we print the instructions at the end of the script?
There was a problem hiding this comment.
Done:
; ./script/linux
....
+ which rustup
+ cat
Note: It's recommended to install and configure Wild or Mold for faster builds.
Run script/install-wild or script/install-mold.
Then add these lines to your ~/.cargo/config.toml:
[target.x86_64-unknown-linux-gnu]
linker = "clang"
rustflags = ["-C", "link-arg=--ld-path=wild"]
[target.aarch64-unknown-linux-gnu]
linker = "clang"
rustflags = ["-C", "link-arg=--ld-path=wild"]
Finished installing Linux dependencies with script/linux
+ exit 0
Definitely having more tests won't hurt, but I doubt you'd see an improvement compared to Mold/Wild: rust-lang/rust#146421 (comment) FYI, David is going to release Wild 0.6 soon. |
|
Great, I see the Wild 0.6 release and have retested with that. That post also has some impressive benchmarks. Strangely the performance of Mold also seems to have improved in benchmarking again today. I'm not sure if a recent OS update improved it dramatically or if there's something wrong with my measurements. I still see a significant improvement in incremental builds but not nearly as much in clean builds. I'll keep looking into it. There's no point adding this devx complication if there's no significant win, but I'm also surprised it's gone away. On the other hand stripping 1s/ 20% off an incremental build, which is reproducible, seems useful in itself. |
|
OK, overall I think this is worth merging, mostly for the improvement in incremental build times. |
| cat <<EOF | ||
| Note: It's recommended to install and configure Wild or Mold for faster builds. | ||
| Run script/install-wild or script/install-mold. | ||
| Then add these lines to your ~/.cargo/config.toml: |
There was a problem hiding this comment.
Everything following Then add these lines should probably live in script/install-wild. Since you might run install-wild after having already done install-linux and forget about those post install steps.
mati865
left a comment
mati865
left a comment
There was a problem hiding this comment.
I didn't test my proposed changes, but I think they should be fine.
Co-authored-by: Mateusz Mikuła <[email protected]>
Co-authored-by: Mateusz Mikuła <[email protected]>
|
Lets ship it! I want those 20% lower compile times :) Thanks again for the PR @sourcefrog! |
Wild changed in 0.6.0 to using gzip rather than xz, and changed the format of the package name. Follows on from and fixes zed-industries#37717
Follows on from #37717 (comment) @dvdsk suggested this but I didn't get to it in the previous PR. # Tested ``` ; sudo rm /usr/local/bin/wild ; ./script/install-wild Downloading from https://github.com/davidlattimore/wild/releases/download/0.6.0/wild-linker-0.6.0-x86_64-unknown-linux-gnu.tar.gz Wild is installed to /usr/local/bin/wild To make it your default, add or merge these lines into your ~/.cargo/config.toml: [target.x86_64-unknown-linux-gnu] linker = "clang" rustflags = ["-C", "link-arg=--ld-path=wild"] [target.aarch64-unknown-linux-gnu] linker = "clang" rustflags = ["-C", "link-arg=--ld-path=wild"] ``` ``` ; sudo rm /usr/local/bin/mold ; ./script/install-mold 2.34.0 Downloading from https://github.com/rui314/mold/releases/download/v2.34.0/mold-2.34.0-x86_64-linux.tar.gz Mold is installed to /usr/local/bin/mold To make it your default, add or merge these lines into your ~/.cargo/config.toml: [target.'cfg(target_os = "linux")'] linker = "clang" rustflags = ["-C", "link-arg=-fuse-ld=mold"] ``` Release Notes: - N/A
4 participants
Summary
Today, Zed uses Mold on Linux, but Wild can be significantly faster.
On my machine, Wild is 14% faster at a whole-tree clean build, 20% faster on an incremental build with a minimal change, and makes no measurable effect on runtime performance of tests.
However, Wild's page says it's not yet ready for production, so it seems to early to switch for production and CI builds.
This PR keeps using Mold in CI and lets developers choose in their own config what linker to use. (The downside of this is that after landing this change, developers will have to do some local config or it will fall back to the default linker which may be slower.)
Wild 0.6 is out, and their announcement has some benchmarks.
cc @davidlattimore from Wild, just fyi
Tasks
.cargo/config.tomlin the treelinux.mdscripts/linuxBenchmarks
I didn't observe any apparent significant change in runtime performance or binary size, or in the in-tree microbenchmarks.
Release Notes: