Performance issues with nx release on large monorepos #33865
Description
Current Behavior
When releasing packages using nx release using conventional commits and independent versions for the packages in a large repository, with 232 projects and hefty git history with several thousand tags, the release process takes more than 30 minutes
Expected Behavior
When releasing package using nx release, it should not take more than 30 minutes to release all package in a big monorepo.
GitHub Repo
No response
Steps to Reproduce
- Create a monorepo with several hundred packages and a big git history with several thousand tags
- add the following release config in
nx.json
{
"release": {
"projectsRelationship": "independent",
"version": {
"conventionalCommits": true,
"versionPrefix": "",
"updateDependents": "always"
},
"changelog": {
"workspaceChangelog": false
},
"groups": {
"packages": {
"projects": ["packages/*"],
"changelog": {
"createRelease": "github"
}
},
"funnels": {
"projects": ["funnels/**"],
"changelog": false
}
},
"releaseTag": {
"pattern": "{projectName}@{version}",
"checkAllBranchesWhen": false
}
}
}- Run pnpm nx release --dry-run
- you can see that it takes a long time to complete a release run
Nx Report
Node : 22.21.1
OS : darwin-arm64
Native Target : aarch64-macos
pnpm : 10.24.0
nx : 22.1.3
lerna : 9.0.3
@nx/js : 22.1.3
@nx/workspace : 22.1.3
@nx/devkit : 22.1.3
typescript : 5.9.3
---------------------------------------
Cache Usage: 56.15 MB / 46.04 GBFailure Logs
Package Manager Version
No response
Operating System
- macOS
- Linux
- Windows
- Other (Please specify)
Additional Information
After some debugging I've found that the functions that take a lot of time per project are getLatestGitTagForPattern and getCommitsRelevantToProjects
getLatestGitTagForPattern is getting tags for each project is iterated when releasing independent versions, in a big repo that could take up to 2 second.
Call Chain
Entry Point (Public API):
releaseChangelog() [changelog.ts]
↓
createReleaseGraph() [release-graph.ts]
↓
ReleaseGraph.init() [release-graph.ts]
↓
resolveCurrentVersionsForProjects() [release-graph.ts]
↓
getLatestGitTagForPattern() [git.ts]
↓
execCommand('git tag ...') [exec-command.ts]
↓
[Back to changelog.ts]
↓
resolveWorkspaceChangelogFromSHA() [version-plan-filtering.ts]
↓
resolveChangelogFromSHA() [version-plan-filtering.ts]
↓
getLatestGitTagForPattern() [git.ts]
↓
execCommand('git tag ...') [exec-command.ts]
↓
[Loop through release groups and projects]
↓
getCachedFromSHA() helper [changelog.ts]
↓
resolveChangelogFromSHA() [version-plan-filtering.ts]
↓
getLatestGitTagForPattern() [git.ts]
↓
execCommand('git tag ...') [exec-command.ts]
getCommitsRelevantToProjects called when calculating the next version of each project when using conventional-commits, each invocation iterate over the commits since previous release and call two functions calculateFileChanges and filterAffected, calling could take several seconds depending on the number of commits and how many files those commits modify, calculateFileChanges take at least 1 - 1.5s when the changed file is a json file.
[PERF] calculateFileChanges: Starting
[PERF] Input: 5 files
[PERF] Filter ignored files: 1.72ms (5 files remaining)
[PERF] Map files: 0.02ms
[PERF] calculateFileChanges: Complete - 1.78ms (5 file changes created)
[PERF] calculateFileChanges: 2.10ms (5 files)
[PERF] Processing file: package.json (.json)
[PERF] Processing JSON file
[PERF] Read base revision: 395.81ms
[PERF] Read head revision: 787.19ms
[PERF] Parse & diff: 1.06ms (2 changes)
[PERF] JSON file complete: 1184.34ms
[PERF] filterAffected: 1193.75ms (232 affected nodes)
[PERF] Processing nodes: 0.04ms (1 relevant projects)
[PERF] Commit f9108432be3: Complete - 1196.08ms
If there are many commits and each commit has many several json files changed, the function could take many seconds to finish.
[PERF] getCommitsRelevantToProjects: Summary
[PERF] Total time: 22966.76ms
[PERF] Commits processed: 209/209 (0 filtered)
[PERF] Cache hits: 0, Cache misses: 209
[PERF] Total calculateFileChanges time: 233.72ms
[PERF] Total filterAffected time: 22691.27ms
[PERF] Total processing nodes time: 4.82ms
[PERF] Relevant commits found: 1 projects affected
Call Chain
Entry Point (Public API):
└── releaseVersion() [version.ts]
↓
createReleaseGraph() [release-graph.ts]
↓
ReleaseGraph.init() [release-graph.ts]
↓ (Step 7: resolveCurrentVersions...)
resolveCurrentVersionsForProjects() [release-graph.ts]
↓ (Fetches git tags & caches)
[Git tags now cached in ReleaseGraph]
↓
[Back to version.ts]
↓
new ReleaseGroupProcessor() [release-group-processor.ts]
↓
processor.processGroups() [release-group-processor.ts]
↓
processGroup() for each group [release-group-processor.ts]
↓
bumpVersions() [release-group-processor.ts]
↓
┌──────────────────────────────────────────┐
│ Fixed Group: │
│ bumpFixedVersionGroup() │ ← Called ONCE per group
│ ↓ │
│ determineVersionBumpForProject() │
│ │
│ Independent Group: │
│ bumpIndependentVersionGroup() │
│ ↓ │
│ for each project { │ ← Called N times
│ determineVersionBumpForProject() │
│ } │
└──────────────────────────────────────────┘
↓
deriveSpecifierFromConventionalCommits() [derive-specifier-from-conventional-commits.ts]
↓
resolveSemverSpecifierFromConventionalCommits() [resolve-semver-specifier.ts]
↓
getGitDiff() [git.ts]
↓
getCommitsRelevantToProjects() [shared.ts]
↓
filterAffected() [affected-project-graph.ts]