perf(plugin): batch hash computation in createNodesV2#866
Open
omer-za wants to merge 1 commit intoPhillip9587:mainfrom
Open
perf(plugin): batch hash computation in createNodesV2#866omer-za wants to merge 1 commit intoPhillip9587:mainfrom
omer-za wants to merge 1 commit intoPhillip9587:mainfrom
Conversation
Replace individual `calculateHashForCreateNodes` calls with a single batched `calculateHashesForCreateNodes` call. This uses `hashMultiGlobWithWorkspaceContext` under the hood, which batches all I/O operations into a single call instead of making N separate calls. In large monorepos (250+ stylelint configs), the per-config hash computation is a significant bottleneck during Nx graph calculation. Made-with: Cursor
Author
|
@Phillip9587 Can you please review it? |
|
View your CI Pipeline Execution ↗ for commit a972d9a
☁️ Nx Cloud last updated this comment at |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace individual
calculateHashForCreateNodescalls with a single batchedcalculateHashesForCreateNodescall in thecreateNodesV2handler.Currently,
createNodesInternalcallscalculateHashForCreateNodesindividually for each config file. Under the hood, each call invokeshashWithWorkspaceContextseparately — meaning N config files trigger N separate I/O-bound hash operations.The
@nx/devkitalready exports a batch counterpart —calculateHashesForCreateNodes— which useshashMultiGlobWithWorkspaceContextto batch all project root hashes into a single call.This PR pre-computes all hashes upfront via the batch API and passes a
hashByRootmap tocreateNodesInternal, falling back to the individual call for the deprecatedcreateNodesv1 path.Benchmark
Measured on a real monorepo with 251 stylelint config files across ~1,600 projects.
Methodology
NX_DAEMON=falseto prevent daemon cachingfind .nx -name "stylelint-*" -delete)performance.now()instrumentation added around thecreateNodesV2handlerChanges
calculateHashesForCreateNodesalongside existingcalculateHashForCreateNodescreateNodesV2, compute all project root hashes upfront via the batch APIhashByRoot: Map<string, string>tocreateNodesInternalcreateNodesInternaluses the map when available, falls back to individual hash for the v1createNodespathMade with Cursor