Start the allocation architecture guide section #121940
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DiannaHohensee
added
>non-issue
:Distributed Coordination/Allocation
Collaborator
|
Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination) |
DaveCTurner
reviewed
Feb 18, 2025
Contributor
DaveCTurner
left a comment
DaveCTurner
left a comment
There was a problem hiding this comment.
Looks good, just a few small comments.
| The `DesiredBalanceShardsAllocator` is what runs shard allocation decisions. It leverages the `DesiredBalanceComputer` to produce | ||
| `DesiredBalance` instances for the cluster based on the latest cluster changes (add/remove nodes, create/remove indices, load, etc). Then | ||
| the `DesiredBalanceReconciler` is invoked to choose the next steps to take to move the cluster from the current shard allocation to the | ||
| latest computed `DesiredBalance` shard allocation. The `Reconciler` will apply changes to a copy of the `RoutingNodes`, which is then |
Contributor
There was a problem hiding this comment.
nit: DesiredBalanceReconciler not just Reconciler (here and below)
| Asynchronous completion of a new `DesiredBalance` will also invoke a reconciliation action, as will cluster state updates completing shard | ||
| moves/recoveries (unthrottling the next shard move/recovery). | ||
|
|
||
| The `ContinuousComputation` maintains a queue of desired balance computation requests, each of which holds the latest cluster information at |
Contributor
There was a problem hiding this comment.
Kind of a queue but really it's only tracking the latest request.
Contributor
Author
There was a problem hiding this comment.
Fixed, thanks for catching that.
I guess the enqueued terminology got stuck in my head :)
| moves/recoveries (unthrottling the next shard move/recovery). | ||
|
|
||
| The `ContinuousComputation` maintains a queue of desired balance computation requests, each of which holds the latest cluster information at | ||
| the time of the request, and a thread that runs the `DesiredBalanceComputer`. The ContinuousComputation thread grabs the latest request, |
Contributor
There was a problem hiding this comment.
nit:
Suggested change
| the time of the request, and a thread that runs the `DesiredBalanceComputer`. The ContinuousComputation thread grabs the latest request, | |
| the time of the request, and a thread that runs the `DesiredBalanceComputer`. The `ContinuousComputation` thread grabs the latest request, |
|
|
||
| There are different priorities in shard allocation, reflected in which moves the `DesiredBalancerReconciler` selects to do first given that | ||
| it can only move, recover, or remove a limited number of shards at once. The first priority is assigning unassigned shards, primaries being | ||
| more important than replicas. The second is to move shards that violate node resource limits or hard limits. The `AllocationDeciders` holds |
Contributor
There was a problem hiding this comment.
violate node resource limits or hard limits
Really, violating any rule as defined by an AllocationDecider.
| There are different priorities in shard allocation, reflected in which moves the `DesiredBalancerReconciler` selects to do first given that | ||
| it can only move, recover, or remove a limited number of shards at once. The first priority is assigning unassigned shards, primaries being | ||
| more important than replicas. The second is to move shards that violate node resource limits or hard limits. The `AllocationDeciders` holds | ||
| a group of `AllocationDecider` types that place hard constraints on shard allocation. There is a decider that manages disk memory usage |
Contributor
There was a problem hiding this comment.
There is a decider
Maybe name these deciders here so folks can go and look them up?
DiannaHohensee
commented
Feb 18, 2025
Contributor
Author
DiannaHohensee
left a comment
DiannaHohensee
left a comment
There was a problem hiding this comment.
Addressed the comments in 2a951eb
| The `DesiredBalanceShardsAllocator` is what runs shard allocation decisions. It leverages the `DesiredBalanceComputer` to produce | ||
| `DesiredBalance` instances for the cluster based on the latest cluster changes (add/remove nodes, create/remove indices, load, etc). Then | ||
| the `DesiredBalanceReconciler` is invoked to choose the next steps to take to move the cluster from the current shard allocation to the | ||
| latest computed `DesiredBalance` shard allocation. The `Reconciler` will apply changes to a copy of the `RoutingNodes`, which is then |
| Asynchronous completion of a new `DesiredBalance` will also invoke a reconciliation action, as will cluster state updates completing shard | ||
| moves/recoveries (unthrottling the next shard move/recovery). | ||
|
|
||
| The `ContinuousComputation` maintains a queue of desired balance computation requests, each of which holds the latest cluster information at |
Contributor
Author
There was a problem hiding this comment.
Fixed, thanks for catching that.
I guess the enqueued terminology got stuck in my head :)
| moves/recoveries (unthrottling the next shard move/recovery). | ||
|
|
||
| The `ContinuousComputation` maintains a queue of desired balance computation requests, each of which holds the latest cluster information at | ||
| the time of the request, and a thread that runs the `DesiredBalanceComputer`. The ContinuousComputation thread grabs the latest request, |
| There are different priorities in shard allocation, reflected in which moves the `DesiredBalancerReconciler` selects to do first given that | ||
| it can only move, recover, or remove a limited number of shards at once. The first priority is assigning unassigned shards, primaries being | ||
| more important than replicas. The second is to move shards that violate node resource limits or hard limits. The `AllocationDeciders` holds | ||
| a group of `AllocationDecider` types that place hard constraints on shard allocation. There is a decider that manages disk memory usage |
|
|
||
| There are different priorities in shard allocation, reflected in which moves the `DesiredBalancerReconciler` selects to do first given that | ||
| it can only move, recover, or remove a limited number of shards at once. The first priority is assigning unassigned shards, primaries being | ||
| more important than replicas. The second is to move shards that violate node resource limits or hard limits. The `AllocationDeciders` holds |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a high-level overview of the main rebalancing components and
how they interact to move shards around the cluster, and decide where
shards should go.
Relates ES-10423
I haven't sold myself on the section titles, if anything better comes to mind.
@DaveCTurner not urgent to review this. Assigning to you because I'm not aware of anyone else already familiar with how this all works.