Skip to content

Fix memory reservation starvation in sort-merge#20642

Open
xudong963 wants to merge 3 commits intoapache:mainfrom
xudong963:fix/sort-merge-reservation-starvation
Open

Fix memory reservation starvation in sort-merge#20642
xudong963 wants to merge 3 commits intoapache:mainfrom
xudong963:fix/sort-merge-reservation-starvation

Conversation

@xudong963
Copy link
Member

@xudong963 xudong963 commented Mar 2, 2026

Which issue does this PR close?

  • Closes #.

Rationale for this change

This PR fixes memory reservation starvation in sort-merge when multiple sort partitions share a GreedyMemoryPool.

When multiple ExternalSorter instances run concurrently and share a single memory pool, the merge phase starves:

  1. Each partition pre-reserves sort_spill_reservation_bytes via merge_reservation
  2. When entering the merge phase, new_empty() was used to create a new reservation starting at 0 bytes, while the pre-reserved bytes sat idle in ExternalSorter.merge_reservation
  3. Those freed bytes were immediately consumed by other partitions racing for memory
  4. The merge could no longer allocate memory from the pool → OOM / starvation

What changes are included in this PR?

Are these changes tested?

I can't find a deterministic way to reproduce the bug, but it occurs in our production.

Are there any user-facing changes?

@github-actions github-actions bot added the physical-plan Changes to the physical-plan crate label Mar 2, 2026
@xudong963 xudong963 marked this pull request as draft March 2, 2026 09:49
@xudong963 xudong963 marked this pull request as ready for review March 2, 2026 10:09
@xudong963
Copy link
Member Author

I can't find a deterministic way to reproduce the bug now, but it occurs in our production. I'd like to get more eyes for the PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant