Fix ResourcesAggregator deadlock with virtual thread executor#6840
Merged
pditommaso merged 1 commit intomasterfrom Feb 18, 2026
Merged
Fix ResourcesAggregator deadlock with virtual thread executor#6840pditommaso merged 1 commit intomasterfrom
pditommaso merged 1 commit intomasterfrom
Conversation
Use a dedicated fixed thread pool in computeSummaryMap() instead of the shared session executor to prevent starvation deadlock when virtual thread carrier threads are saturated. Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
✅ Deploy Preview for nextflow-docs-staging canceled.
|
bentsherman
approved these changes
Feb 18, 2026
Member
bentsherman
left a comment
There was a problem hiding this comment.
The analysis is slightly off -- PublishDir doesn't use the main session execService to publish files -- but I think the change is still reasonable
Member
Author
|
I think it get confused with the similar one #6833, but yes, it's correct |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix a deadlock in
ResourcesAggregator.computeSummaryMap()that causes Nextflow to hang indefinitely during shutdown when generating the execution report.Problem
ResourcesAggregatorused the shared sessionExecutorService— aThreadPerTaskExecutorbacked by virtual threads — to parallelize summary stats computation viainvokeAll().When other virtual threads (e.g.
PublishDirfile transfers) saturate or pin all ForkJoinPool carrier threads, the new tasks submitted byinvokeAllcan never be scheduled. SinceinvokeAllblocks until all futures complete, this creates a starvation deadlock: the main thread waits forever atcomputeSummaryMap()duringonFlowComplete.Observed in production — thread dump shows the main thread stuck at:
This is related to but independent of #6833 — that PR fixes one cause of carrier thread saturation (S3 delete), but
ResourcesAggregatoris vulnerable to any scenario where carrier threads are busy.Fix
Replace the shared session virtual thread executor with a dedicated
FixedThreadPoolscoped to thecomputeSummaryMap()call. This is CPU-bound computation that:This also removes the
Sessiondependency fromResourcesAggregator, simplifying both the class and its tests.Test plan
ResourcesAggregatorTesttests pass (summary computation, JSON rendering, insertion order)ReportObserverTestpassesnf-towerplugin compiles cleanly🤖 Generated with Claude Code