Skip to content

perf: use a dense ModuleGraph connection rollback journal#14745

Draft
hardfist wants to merge 1 commit into
mainfrom
codex/compact-module-graph-connection-overlay
Draft

perf: use a dense ModuleGraph connection rollback journal#14745
hardfist wants to merge 1 commit into
mainfrom
codex/compact-module-graph-connection-overlay

Conversation

@hardfist

@hardfist hardfist commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • apply checkpointed connection mutations directly to the dense base vector so every connection read keeps the fast single-vector lookup path
  • record each dependency's original connection only once in a compact undo journal
  • use a one-byte changed marker per covered dependency ID and store values only for connections that are actually modified
  • restore original values and truncate newly added dependency slots when the ModuleGraph checkpoint is reset
  • cover insert, remove, repeated write, repeated mutable access, and sparse dependency rollback behavior

This targets the high-cardinality dependency path in ModuleGraph. Connection reads heavily outnumber checkpoint mutations during module concatenation, so moving rollback work to the mutation/reset side removes the overlay branch and secondary value lookup from the hot read path.

The focused rust@create_concatenate_module Callgrind stage improved from 47,436,218 to 47,026,994 instructions on the same local build path: -409,224 Ir (-0.86%). Three consecutive final measurements produced the same result. Compared with the previous compact-overlay implementation in this PR, the journal removes 512,454 Ir (-1.08%).

Checked with six focused Rust rollback tests, pnpm run build:cli:dev, pnpm run test:unit (46 files and 8,787 tests passed), Rust/JS formatting, and cargo clippy -p rspack_core --all-targets. The workspace-wide all-features Clippy command remains blocked by the existing rkyv 0.7/0.8 conflict in rspack_cacheable.

Related links

Checklist

  • Tests updated.
  • Documentation is not required for this internal data structure change.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

📦 Binary Size-limit

Comparing 03b8224 to refactor(cli): use @rstackjs/load-config (#14738) by Jiahan Chen

🎉 Size decreased by 4.00KB from 66.42MB to 66.42MB (⬇️0.01%)

@codspeed-hq

codspeed-hq Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will improve performance by 4.49%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 2 regressed benchmarks
✅ 39 untouched benchmarks
⏩ 47 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation rust@create_named_chunk_ids 2.9 ms 3 ms -2.45%
Simulation rust@create_chunk_ids 3 ms 3.1 ms -2.23%
Simulation rust@module_graph_api 3.5 ms 2.9 ms +22.11%
Simulation rust@create_named_module_ids 3.2 ms 3.1 ms +2.36%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing codex/compact-module-graph-connection-overlay (03b8224) with main (95c928f)

Open in CodSpeed

Footnotes

  1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@hardfist hardfist force-pushed the codex/compact-module-graph-connection-overlay branch from ad116a8 to 03b8224 Compare July 9, 2026 22:09
@hardfist hardfist changed the title perf: compact module graph connection overlay perf: use a dense ModuleGraph connection rollback journal Jul 9, 2026
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

📝 Benchmark detail: Open

Name Base (95c928f) Current Change
10000_big_production-mode_disable-minimize + exec 12 s ± 176 ms 11.9 s ± 186 ms -0.55 %
10000_development-mode + exec 896 ms ± 21 ms 874 ms ± 21 ms -2.48 %
10000_development-mode_hmr + stats 163 ms ± 9.7 ms 154 ms ± 5.4 ms -5.85 %
10000_development-mode_noop-loader + exec 1.78 s ± 140 ms 1.72 s ± 25 ms -3.20 %
10000_production-mode + exec 1.03 s ± 49 ms 997 ms ± 41 ms -3.19 %
10000_production-mode_persistent-cold + exec 1.19 s ± 23 ms 1.15 s ± 14 ms -3.86 %
10000_production-mode_persistent-hot + exec 643 ms ± 10 ms 611 ms ± 24 ms -5.01 %
10000_production-mode_source-map + exec 1.19 s ± 24 ms 1.14 s ± 23 ms -4.22 %
arco-pro_development-mode + exec 1.3 s ± 93 ms 1.31 s ± 87 ms +0.65 %
arco-pro_development-mode_hmr + stats 33 ms ± 1.2 ms 32 ms ± 0.36 ms -5.12 %
arco-pro_production-mode + exec 2.37 s ± 153 ms 2.29 s ± 78 ms -3.46 %
arco-pro_production-mode_generate-package-json-webpack-plugin + exec 2.44 s ± 62 ms 2.34 s ± 136 ms -3.98 %
arco-pro_production-mode_persistent-cold + exec 2.39 s ± 99 ms 2.34 s ± 126 ms -2.06 %
arco-pro_production-mode_persistent-hot + exec 348 ms ± 9.8 ms 341 ms ± 4.2 ms -2.01 %
arco-pro_production-mode_source-map + exec 2.84 s ± 126 ms 2.81 s ± 97 ms -1.03 %
arco-pro_production-mode_source-map_persistent-cold + exec 2.97 s ± 146 ms 2.9 s ± 115 ms -2.48 %
arco-pro_production-mode_source-map_persistent-hot + exec 479 ms ± 15 ms 467 ms ± 11 ms -2.42 %
arco-pro_production-mode_traverse-chunk-modules + exec 2.37 s ± 41 ms 2.31 s ± 170 ms -2.85 %
bundled-threejs_development-mode + exec 182 ms ± 7.5 ms 176 ms ± 3.6 ms -3.40 %
bundled-threejs_production-mode + exec 212 ms ± 8.2 ms 206 ms ± 4.9 ms -2.55 %
large-dyn-imports_development-mode + exec 1.17 s ± 31 ms 1.15 s ± 18 ms -1.55 %
large-dyn-imports_production-mode + exec 1.26 s ± 38 ms 1.25 s ± 38 ms -0.95 %
threejs_development-mode_10x + exec 782 ms ± 9.5 ms 771 ms ± 24 ms -1.51 %
threejs_development-mode_10x_hmr + stats 114 ms ± 1.6 ms 107 ms ± 2.9 ms -6.00 %
threejs_production-mode_10x + exec 2.88 s ± 162 ms 2.78 s ± 83 ms -3.41 %
threejs_production-mode_10x_persistent-cold + exec 2.97 s ± 33 ms 2.88 s ± 47 ms -2.86 %
threejs_production-mode_10x_persistent-hot + exec 380 ms ± 8.7 ms 366 ms ± 7.6 ms -3.74 %
threejs_production-mode_10x_source-map + exec 3.66 s ± 204 ms 3.56 s ± 41 ms -2.58 %
10000_big_production-mode_disable-minimize + rss memory 2181 MiB ± 51.9 MiB 2208 MiB ± 28.5 MiB +1.21 %
10000_development-mode + rss memory 544 MiB ± 4.91 MiB 543 MiB ± 7.34 MiB -0.22 %
10000_development-mode_hmr + rss memory 792 MiB ± 40.3 MiB 779 MiB ± 65.3 MiB -1.64 %
10000_development-mode_noop-loader + rss memory 815 MiB ± 17.7 MiB 809 MiB ± 14.1 MiB -0.71 %
10000_production-mode + rss memory 466 MiB ± 7.76 MiB 469 MiB ± 4.61 MiB +0.50 %
10000_production-mode_persistent-cold + rss memory 661 MiB ± 4.07 MiB 660 MiB ± 8.8 MiB -0.18 %
10000_production-mode_persistent-hot + rss memory 659 MiB ± 12 MiB 633 MiB ± 26.8 MiB -4.02 %
10000_production-mode_source-map + rss memory 480 MiB ± 21.5 MiB 455 MiB ± 76.1 MiB -5.38 %
arco-pro_development-mode + rss memory 401 MiB ± 1.84 MiB 397 MiB ± 4.21 MiB -0.84 %
arco-pro_development-mode_hmr + rss memory 422 MiB ± 12.5 MiB 419 MiB ± 7.84 MiB -0.65 %
arco-pro_production-mode + rss memory 578 MiB ± 7.59 MiB 578 MiB ± 8.15 MiB -0.02 %
arco-pro_production-mode_generate-package-json-webpack-plugin + rss memory 592 MiB ± 9.63 MiB 579 MiB ± 78.7 MiB -2.25 %
arco-pro_production-mode_persistent-cold + rss memory 665 MiB ± 30.9 MiB 662 MiB ± 17.9 MiB -0.36 %
arco-pro_production-mode_persistent-hot + rss memory 292 MiB ± 9.99 MiB 294 MiB ± 6.98 MiB +0.74 %
arco-pro_production-mode_source-map + rss memory 676 MiB ± 18.6 MiB 681 MiB ± 12.2 MiB +0.79 %
arco-pro_production-mode_source-map_persistent-cold + rss memory 808 MiB ± 8.34 MiB 809 MiB ± 28.6 MiB +0.21 %
arco-pro_production-mode_source-map_persistent-hot + rss memory 417 MiB ± 11.1 MiB 421 MiB ± 7.52 MiB +1.06 %
arco-pro_production-mode_traverse-chunk-modules + rss memory 577 MiB ± 9.53 MiB 578 MiB ± 6.1 MiB +0.20 %
bundled-threejs_development-mode + rss memory 127 MiB ± 4.75 MiB 126 MiB ± 6.05 MiB -0.67 %
bundled-threejs_production-mode + rss memory 139 MiB ± 4.05 MiB 141 MiB ± 3.76 MiB +1.47 %
large-dyn-imports_development-mode + rss memory 553 MiB ± 6.03 MiB 559 MiB ± 3.34 MiB +1.05 %
large-dyn-imports_production-mode + rss memory 403 MiB ± 5.82 MiB 407 MiB ± 3.17 MiB +0.80 %
threejs_development-mode_10x + rss memory 435 MiB ± 9.21 MiB 437 MiB ± 23 MiB +0.30 %
threejs_development-mode_10x_hmr + rss memory 595 MiB ± 16.7 MiB 599 MiB ± 16.5 MiB +0.66 %
threejs_production-mode_10x + rss memory 610 MiB ± 2.44 MiB 612 MiB ± 10.4 MiB +0.31 %
threejs_production-mode_10x_persistent-cold + rss memory 576 MiB ± 11.3 MiB 574 MiB ± 9.83 MiB -0.31 %
threejs_production-mode_10x_persistent-hot + rss memory 368 MiB ± 14 MiB 366 MiB ± 10.2 MiB -0.41 %
threejs_production-mode_10x_source-map + rss memory 742 MiB ± 21.8 MiB 708 MiB ± 56 MiB -4.55 %

Base persistent cache hit rate: 👍

Current persistent cache hit rate: 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant