Double close on shuffle during field loading #130838
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes a bug during field loading where we could double-close blocks if we failed to allocate memory during the un-shuffling portion of field loading from single segments. Unit test incoming in the followup. Closes elastic#130426 Closes elastic#130790 Closes elastic#130791 Closes elastic#130792 Closes elastic#130793 Closes elastic#130270 Closes elastic#130788 Closes elastic#130122 Closes elastic#130827
nik9000
added
>non-issue
auto-backport
elasticsearchmachine
added
the
Team:Analytics
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
nik9000
commented
Jul 8, 2025
| assumeFalse("strange exception in the test, fix soon", true); | ||
| return ByteSizeValue.ofKb(1); | ||
| } | ||
|
|
There was a problem hiding this comment.
This test didn't catch any failures, but it would have if we'd made a non-"simple" version. I'll expand this in a followup I think.
| List<Page> input = CannedSourceOperator.collectPages(simpleInput(inputFactoryContext.blockFactory(), between(1_000, 10_000))); | ||
|
|
||
| ByteSizeValue memoryLimitForSimple = enoughMemoryForSimple(); | ||
| Operator.OperatorFactory simple = simple(new SimpleOptions(true)); |
There was a problem hiding this comment.
This shift fixed the test.
| public int count() { | ||
| return docs.getPositionCount(); | ||
| } | ||
| Block[] unshuffled = new Block[target.length]; |
There was a problem hiding this comment.
Here's the bug - you can't use the target array and then shuffle into it.
This was referenced Jul 8, 2025
|
Merging despite the serverless error. It's unrelated and this blocks a bunch of spurious failures. |
💔 Backport failed
You can use sqren/backport to manually backport by running |
nik9000
added a commit
to nik9000/elasticsearch
that referenced
this pull request
Jul 8, 2025
Fixes a bug during field loading where we could double-close blocks if we failed to allocate memory during the un-shuffling portion of field loading from single segments. Unit test incoming in the followup. Closes elastic#130426 Closes elastic#130790 Closes elastic#130791 Closes elastic#130792 Closes elastic#130793 Closes elastic#130270 Closes elastic#130788 Closes elastic#130122 Closes elastic#130827
|
9.1 backport: #130848 |
nik9000
added a commit
to nik9000/elasticsearch
that referenced
this pull request
Jul 8, 2025
Fixes a bug during field loading where we could double-close blocks if we failed to allocate memory during the un-shuffling portion of field loading from single segments. Unit test incoming in the followup. Closes elastic#130426 Closes elastic#130790 Closes elastic#130791 Closes elastic#130792 Closes elastic#130793 Closes elastic#130270 Closes elastic#130788 Closes elastic#130122 Closes elastic#130827
nik9000
added a commit
that referenced
this pull request
Jul 8, 2025
Fixes a bug during field loading where we could double-close blocks if we failed to allocate memory during the un-shuffling portion of field loading from single segments. Unit test incoming in the followup. Closes #130426 Closes #130790 Closes #130791 Closes #130792 Closes #130793 Closes #130270 Closes #130788 Closes #130122 Closes #130827
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes a bug during field loading where we could double-close blocks if we failed to allocate memory during the un-shuffling portion of field loading from single segments.
Unit test incoming in the followup.
Closes #130426
Closes #130790
Closes #130791
Closes #130792
Closes #130793
Closes #130270
Closes #130788
Closes #130122
Closes #130827
Closes #130831