feat(core): use jemalloc with tuned decay timers for native module#34444
feat(core): use jemalloc with tuned decay timers for native module#34444leosvelperez wants to merge 3 commits intomasterfrom
Conversation
✅ Deploy Preview for nx-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for nx-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
View your CI Pipeline Execution ↗ for commit 647ee9d
☁️ Nx Cloud last updated this comment at |
288ca83 to
bda7b28
Compare
|
Failed to publish a PR release of this pull request, triggered by @leosvelperez. |
5664423 to
bfe5813
Compare
|
Failed to publish a PR release of this pull request, triggered by @leosvelperez. |
cde6c7d to
ae6bae4
Compare
753d4c9 to
d77d7aa
Compare
There was a problem hiding this comment.
Nx Cloud is proposing a fix for your failed CI:
These changes adjust the jemalloc configuration to fix Gradle daemon termination failures in e2e tests. The original muzzy_decay_ms:0 setting was too aggressive, causing immediate memory unmapping via MADV_DONTNEED that interfered with Node.js process coordination when managing multiple Gradle daemon subprocesses. Setting muzzy_decay_ms:1000 maintains the memory reduction benefits while allowing a brief delay before aggressive unmapping, preventing interference with concurrent process management and signal handling.
Tip
✅ We verified this fix by re-running e2e-gradle:e2e-ci--src/gradle-plugin-v1.test.ts.
diff --git a/.cargo/config.toml b/.cargo/config.toml
index 2e9a2981dd..93c18852e5 100644
--- a/.cargo/config.toml
+++ b/.cargo/config.toml
@@ -1,5 +1,5 @@
[env]
-JEMALLOC_SYS_WITH_MALLOC_CONF = "dirty_decay_ms:1000,muzzy_decay_ms:0"
+JEMALLOC_SYS_WITH_MALLOC_CONF = "dirty_decay_ms:1000,muzzy_decay_ms:1000"
[build]
target-dir = 'dist/target'
Or Apply changes locally with:
npx nx-cloud apply-locally 85eG-u8ne
Apply fix locally with your editor ↗ View interactive diff ↗
🎓 Learn more about Self-Healing CI on nx.dev
Current Behavior
The Nx native module (Rust cdylib loaded by Node.js) uses the system allocator. The daemon process retains a large RSS footprint after the initial project graph build, even though most of that memory is no longer in use. On macOS and Linux, the system allocator doesn't aggressively return freed pages to the OS.
Expected Behavior
The daemon's steady-state RSS drops significantly after graph build by using jemalloc with tuned page purge timers. Peak RSS and wall time are unaffected.
Changes
Adds tikv-jemallocator as the global allocator on Linux and macOS, with two compile-time settings:
dirty_decay_ms:1000— returns freed pages to the OS after 1s instead of the default 10s. Tuned to Nx's phase-separated workload (graph build → idle → task execution), where transitions happen every ~30-60s. Benchmarked against 5s and 10s — both too slow to purge between phases.muzzy_decay_ms:0— skips the lazy purge phase (MADV_FREE) and goes straight toMADV_DONTNEED. Required on macOS and Linux ≥ 4.5 whereMADV_FREEdoesn't actually reduce RSS.Windows and WASI continue using the system allocator. Windows is excluded because
tikv-jemalloc-sysfails to build with MSVC (spaces incl.exepath break the autoconf configure script). Tracked upstream in tikv/jemallocator#99.Other Settings Considered
Tested narenas reduction, tcache_max, extent fit tuning, background threads, and decay timer values. Only the decay timer configuration improved steady-state RSS without wall time regression.