-
Notifications
You must be signed in to change notification settings - Fork 25.6k
Calculate routing num shards correctly during reshard #125601
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
ankikuma
merged 21 commits into
elastic:main
from
ankikuma:03132025/ReshardRoutingNumShards
May 8, 2025
Merged
Changes from 12 commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
671c348
commit
ankikuma 7ae62e7
indexMetadata comments
ankikuma 31281e5
Merge remote-tracking branch 'upstream/main' into 03132025/ReshardRou…
ankikuma ff98f8e
SplitIndexIT
ankikuma 37d61fd
Merge remote-tracking branch 'upstream/main' into 03132025/ReshardRou…
ankikuma b29a437
IndexMetadata.java
ankikuma eacb67a
[CI] Auto commit changes from spotless
0a15de3
commit
ankikuma 86f66bc
Merge remote-tracking branch 'upstream/main' into 03132025/ReshardRou…
ankikuma 41a70ba
merge conflicts in elasticsearch
ankikuma 44c9c4f
merge conflicts in elasticsearch
ankikuma c8382c2
merge conflicts in elasticsearch
ankikuma e77399d
commit elasticsearch
ankikuma 2946f37
Merge remote-tracking branch 'upstream/main' into 03132025/ReshardRou…
ankikuma c21b9f5
Merge remote-tracking branch 'upstream/main' into 03132025/ReshardRou…
ankikuma 33a8b45
Merge remote-tracking branch 'upstream/main' into 03132025/ReshardRou…
ankikuma a2de378
address review comments and add test
ankikuma 8d10925
Merge remote-tracking branch 'upstream/main' into 03132025/ReshardRou…
ankikuma fa25099
reshardAddShards
ankikuma 4886ff3
Merge remote-tracking branch 'upstream/main' into 03132025/ReshardRou…
ankikuma 47ab570
minor change
ankikuma File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changing the API this way makes it possible to accidentally supply the wrong metadata to the method, and we know we've already provided the right metadata in the builder's constructor.
I think the reason you're passing it in is that
getIndexNumberOfRoutingShardswants a metadata object but we've already decomposed it into parts in the constructor. I think it's probably fine to just hold on to a reference to the whole thing for the life of the builder (i.e.,this.indexMetadata = indexMetadatain the constructor or something), or to change the interface togetIndexNumberOfRoutingShards, which only has a handful of users that mostly pass in null. Maybe the first option is the simplest?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, I didn't like passing in the sourceMetadata either. But I don't know if I want to hold onto a reference to the whole thing because it looks like we would end up with some kind of recursion in toXContent() wouldn't we ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right, I hadn't noticed that when IndexMetadata goes over the wire it passes through the Builder interface.
Looking at
MetadataCreateIndexService::getIndexNumberOfRoutingShardsthe only thing it actually usessourceMetadatafor is to callgetNumberOfShardson it if it exists. So one option to avoid passing in this essentially redundantsourceMetadatafield would be to refactorgetIndexNumberOfRoutingShardsa bit to have an inner method that just takesroutingNumShardsor 0 if metadata is null, which you could call directly from here, and then make the existinggetIndexNumberOfRoutingShards(Settings indexSettings, @Nullable IndexMetadata sourceMetadata)just be something likereturn getIndexNumberOfRoutingShards(settings, sourceMetadata == null ? 0 : sourceMetadata.getRoutingNumShards()There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes sure, I refactored the code.