|
| 1 | +[[shard-allocation-relocation-recovery]] |
| 2 | +=== Shard allocation, relocation, and recovery |
| 3 | + |
| 4 | +Each <<documents-indices,index>> in Elasticsearch is divided into one or more <<scalability,shards>>. |
| 5 | +Each document in an index belongs to a single shard. |
| 6 | + |
| 7 | +A cluster can contain multiple copies of a shard. Each shard has one distinguished shard copy called the _primary_, and zero or more non-primary copies called _replicas_. The primary shard copy serves as the main entry point for all indexing operations. The operations on the primary shard copy are then forwarded to its replicas. |
| 8 | + |
| 9 | +Replicas maintain redundant copies of your data across the <<modules-node,nodes>> in your cluster, protecting against hardware failure and increasing capacity to serve read requests like searching or retrieving a document. If the primary shard copy fails, then a replica is promoted to primary and takes over the primary's responsibilities. |
| 10 | + |
| 11 | +Over the course of normal operation, Elasticsearch allocates shard copies to nodes, relocates shard copies across nodes to balance the cluster or satisfy new allocation constraints, and recovers shards to initialize new copies. In this topic, you'll learn how these operations work and how you can control them. |
| 12 | + |
| 13 | +TIP: To learn about optimizing the number and size of shards in your cluster, refer to <<size-your-shards,Size your shards>>. To learn about how read and write operations are replicated across shards and shard copies, refer to <<docs-replication,Reading and writing documents>>. |
| 14 | + |
| 15 | +[[shard-allocation]] |
| 16 | +==== Shard allocation |
| 17 | + |
| 18 | +include::{es-ref-dir}/modules/shard-allocation-desc.asciidoc[] |
| 19 | + |
| 20 | +By default, the primary and replica shard copies for an index can be allocated to any node in the cluster, and may be relocated to rebalance the cluster. |
| 21 | + |
| 22 | +===== Adjust shard allocation settings |
| 23 | + |
| 24 | +You can control how shard copies are allocated using the following settings: |
| 25 | + |
| 26 | +- <<modules-cluster,Cluster-level shard allocation settings>>: Use these settings to control how shard copies are allocated and balanced across the entire cluster. For example, you might want to allocate nodes availability zones, or prevent certain nodes from being used so you can perform maintenance. |
| 27 | + |
| 28 | +- <<index-modules-allocation,Index-level shard allocation settings>>: Use these settings to control how the shard copies for a specific index are allocated. For example, you might want to allocate an index to a node in a specific data tier, or to an node with specific attributes. |
| 29 | + |
| 30 | +===== Monitor shard allocation |
| 31 | + |
| 32 | +If a shard copy is unassigned, it means that the shard copy is not allocated to any node in the cluster. This can happen if there are not enough nodes in the cluster to allocate the shard copy, or if the shard copy can't be allocated to any node that satisfies the shard allocation filtering rules. When a shard copy is unassigned, your cluster is considered unhealthy and returns a yellow or red cluster health status. |
| 33 | + |
| 34 | +You can use the following APIs to monitor shard allocation: |
| 35 | + |
| 36 | +- <<cluster-allocation-explain,Cluster allocation explain>> |
| 37 | +- <<cat-allocation,cat allocation>> |
| 38 | +- <<cluster-health,cluster health>> |
| 39 | + |
| 40 | +<<red-yellow-cluster-status,Learn more about troubleshooting unassigned shard copies and recovering your cluster health>>. |
| 41 | + |
| 42 | +[[shard-recovery]] |
| 43 | +==== Shard recovery |
| 44 | + |
| 45 | +include::{es-ref-dir}/modules/shard-recovery-desc.asciidoc[] |
| 46 | + |
| 47 | +===== Adjust shard recovery settings |
| 48 | + |
| 49 | +To control how shards are recovered, for example the resources that can be used by recovery operations, and which indices should be prioritized for recovery, you can adjust the following settings: |
| 50 | + |
| 51 | +- <<recovery,Index recovery settings>> |
| 52 | +- <<modules-cluster,Cluster-level shard allocation settings>> |
| 53 | +- <<index-modules-allocation,Index-level shard allocation settings>>, including <<delayed-allocation,delayed allocation>> and <<recovery-prioritization,index recovery prioritization>> |
| 54 | + |
| 55 | +Shard recovery operations also respect general shard allocation settings. |
| 56 | + |
| 57 | +===== Monitor shard recovery |
| 58 | + |
| 59 | +You can use the following APIs to monitor shard allocation: |
| 60 | + |
| 61 | + - View a list of in-progress and completed recoveries using the <<cat-recovery,cat recovery API>> |
| 62 | + - View detailed information about a specific recovery using the <<indices-recovery,index recovery API>> |
| 63 | + |
| 64 | +[[shard-relocation]] |
| 65 | +==== Shard relocation |
| 66 | + |
| 67 | +Shard relocation is the process of moving shard copies from one node to another. This can happen when a node joins or leaves the cluster, or when the cluster is rebalancing. |
| 68 | + |
| 69 | +When a shard copy is relocated, it is created as a new shard copy on the target node. When the shard copy is fully allocated and recovered, the old shard copy is deleted. If the shard copy being relocated is a primary, then the new shard copy is marked as primary before the old shard copy is deleted. |
| 70 | + |
| 71 | +===== Adjust shard relocation settings |
| 72 | + |
| 73 | +You can control how and when shard copies are relocated. For example, you can adjust the rebalancing settings that control when shard copies are relocated to balance the cluster, or the high watermark for disk-based shard allocation that can trigger relocation. These settings are part of the <<modules-cluster,cluster-level shard allocation settings>>. |
| 74 | + |
| 75 | +Shard relocation operations also respect shard allocation and recovery settings. |
0 commit comments