Skip to content

Commit 80a113c

Browse files
committed
more fixes to pages
1 parent 4514f2a commit 80a113c

File tree

9 files changed

+22
-246
lines changed

9 files changed

+22
-246
lines changed

deploy-manage/distributed-architecture/discovery-cluster-formation/discovery-hosts-providers.md

Lines changed: 2 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@ By default the cluster formation module offers two seed hosts providers to confi
2525

2626
Each seed hosts provider yields the IP addresses or hostnames of the seed nodes. If it returns any hostnames then these are resolved to IP addresses using a DNS lookup. If a hostname resolves to multiple IP addresses then {{es}} tries to find a seed node at all of these addresses. If the hosts provider does not explicitly give the TCP port of the node by then, it will implicitly use the first port in the port range given by `transport.profiles.default.port`, or by `transport.port` if `transport.profiles.default.port` is not set. The number of concurrent lookups is controlled by `discovery.seed_resolver.max_concurrent_resolvers` which defaults to `10`, and the timeout for each lookup is controlled by `discovery.seed_resolver.timeout` which defaults to `5s`. Note that DNS lookups are subject to [JVM DNS caching](../../deploy/self-managed/networkaddress-cache-ttl.md).
2727

28-
2928
#### Settings-based seed hosts provider [settings-based-hosts-provider]
3029

3130
The settings-based seed hosts provider uses a node setting to configure a static list of the addresses of the seed nodes. These addresses can be given as hostnames or IP addresses; hosts specified as hostnames are resolved to IP addresses during each round of discovery.
@@ -42,8 +41,6 @@ discovery.seed_hosts:
4241
1. The port will default to `transport.profiles.default.port` and fallback to `transport.port` if not specified.
4342
2. If a hostname resolves to multiple IP addresses, {{es}} will attempt to connect to every resolved address.
4443

45-
46-
4744
#### File-based seed hosts provider [file-based-hosts-provider]
4845

4946
The file-based seed hosts provider configures a list of hosts via an external file. {{es}} reloads this file when it changes, so that the list of seed nodes can change dynamically without needing to restart each node. For example, this gives a convenient mechanism for an {{es}} instance that is run in a Docker container to be dynamically supplied with a list of IP addresses to connect to when those IP addresses may not be known at node startup.
@@ -74,19 +71,14 @@ Host names are allowed instead of IP addresses and are resolved by DNS as descri
7471

7572
You can also add comments to this file. All comments must appear on their lines starting with `#` (i.e. comments cannot start in the middle of a line).
7673

77-
78-
#### EC2 hosts provider [ec2-hosts-provider]
74+
#### EC2 hosts provider [ec2-hosts-provider]
7975

8076
The [EC2 discovery plugin](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch-plugins/discovery-ec2.md) adds a hosts provider that uses the [AWS API](https://github.com/aws/aws-sdk-java) to find a list of seed nodes.
8177

82-
83-
#### Azure Classic hosts provider [azure-classic-hosts-provider]
78+
#### Azure Classic hosts provider [azure-classic-hosts-provider]
8479

8580
The [Azure Classic discovery plugin](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch-plugins/discovery-azure-classic.md) adds a hosts provider that uses the Azure Classic API find a list of seed nodes.
8681

87-
8882
#### Google Compute Engine hosts provider [gce-hosts-provider]
8983

9084
The [GCE discovery plugin](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch-plugins/discovery-gce.md) adds a hosts provider that uses the GCE API find a list of seed nodes.
91-
92-

deploy-manage/distributed-architecture/discovery-cluster-formation/modules-discovery-bootstrap-cluster.md

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@ If you leave `cluster.initial_master_nodes` in place once the cluster has formed
2525

2626
::::
2727

28-
2928
The simplest way to create a new cluster is for you to select one of your master-eligible nodes that will bootstrap itself into a single-node cluster, which all the other nodes will then join. This simple approach is not resilient to failures until the other master-eligible nodes have joined the cluster. For example, if you have a master-eligible node with [node name](../../deploy/self-managed/important-settings-configuration.md#node-name) `master-a` then configure it as follows (omitting `cluster.initial_master_nodes` from the configuration of all other nodes):
3029

3130
```yaml
@@ -45,7 +44,6 @@ cluster.initial_master_nodes:
4544
You must set `cluster.initial_master_nodes` to the same list of nodes on each node on which it is set in order to be sure that only a single cluster forms during bootstrapping. If `cluster.initial_master_nodes` varies across the nodes on which it is set then you may bootstrap multiple clusters. It is usually not possible to recover from this situation without losing data.
4645
::::
4746

48-
4947
::::{admonition} Node name formats must match
5048
:name: modules-discovery-bootstrap-cluster-fqdns
5149

@@ -62,12 +60,10 @@ This message shows the node names `master-a.example.com` and `master-b.example.c
6260

6361
::::
6462

65-
6663
## Choosing a cluster name [bootstrap-cluster-name]
6764

6865
The [`cluster.name`](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/miscellaneous-cluster-settings.md#cluster-name) setting enables you to create multiple clusters which are separated from each other. Nodes verify that they agree on their cluster name when they first connect to each other, and Elasticsearch will only form a cluster from nodes that all have the same cluster name. The default value for the cluster name is `elasticsearch`, but it is recommended to change this to reflect the logical name of the cluster.
6966

70-
7167
## Auto-bootstrapping in development mode [bootstrap-auto-bootstrap]
7268

7369
By default each node will automatically bootstrap itself into a single-node cluster the first time it starts. If any of the following settings are configured then auto-bootstrapping will not take place:
@@ -100,6 +96,3 @@ If you intended to form a new multi-node cluster but instead bootstrapped a coll
10096
6. Remove `cluster.initial_master_nodes` from every node’s configuration.
10197

10298
::::
103-
104-
105-

deploy-manage/distributed-architecture/discovery-cluster-formation/modules-discovery-quorums.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,12 @@ If you stop half or more of the nodes in the voting configuration at the same ti
2020

2121
::::
2222

23-
2423
After a master-eligible node has joined or left the cluster the elected master may issue a cluster-state update that adjusts the voting configuration to match, and this can take a short time to complete. It is important to wait for this adjustment to complete before removing more nodes from the cluster. See [Removing master-eligible nodes](../../maintenance/add-and-remove-elasticsearch-nodes.md#modules-discovery-removing-nodes) for more information.
2524

26-
2725
## Master elections [_master_elections]
2826

2927
Elasticsearch uses an election process to agree on an elected master node, both at startup and if the existing elected master fails. Any master-eligible node can start an election, and normally the first election that takes place will succeed. Elections only usually fail when two nodes both happen to start their elections at about the same time, so elections are scheduled randomly on each node to reduce the probability of this happening. Nodes will retry elections until a master is elected, backing off on failure, so that eventually an election will succeed (with arbitrarily high probability). The scheduling of master elections are controlled by the [master election settings](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/discovery-cluster-formation-settings.md#master-election-settings).
3028

31-
3229
## Cluster maintenance, rolling restarts and migrations [_cluster_maintenance_rolling_restarts_and_migrations]
3330

3431
Many cluster maintenance tasks involve temporarily shutting down one or more nodes and then starting them back up again. By default {{es}} can remain available if one of its master-eligible nodes is taken offline, such as during a rolling upgrade. Furthermore, if multiple nodes are stopped and then started again then it will automatically recover, such as during a full cluster restart. There is no need to take any further action with the APIs described here in these cases, because the set of master nodes is not changing permanently.

deploy-manage/distributed-architecture/discovery-cluster-formation/modules-discovery-voting.md

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,6 @@ GET /_cluster/state?filter_path=metadata.cluster_coordination.last_committed_con
3131
The current voting configuration is not necessarily the same as the set of all available master-eligible nodes in the cluster. Altering the voting configuration involves taking a vote, so it takes some time to adjust the configuration as nodes join or leave the cluster. Also, there are situations where the most resilient configuration includes unavailable nodes or does not include some available nodes. In these situations, the voting configuration differs from the set of available master-eligible nodes in the cluster.
3232
::::
3333

34-
3534
Larger voting configurations are usually more resilient, so Elasticsearch normally prefers to add master-eligible nodes to the voting configuration after they join the cluster. Similarly, if a node in the voting configuration leaves the cluster and there is another master-eligible node in the cluster that is not in the voting configuration then it is preferable to swap these two nodes over. The size of the voting configuration is thus unchanged but its resilience increases.
3635

3736
It is not so straightforward to automatically remove nodes from the voting configuration after they have left the cluster. Different strategies have different benefits and drawbacks, so the right choice depends on how the cluster will be used. You can control whether the voting configuration automatically shrinks by using the [`cluster.auto_shrink_voting_configuration` setting](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/discovery-cluster-formation-settings.md).
@@ -40,19 +39,16 @@ It is not so straightforward to automatically remove nodes from the voting confi
4039
If `cluster.auto_shrink_voting_configuration` is set to `true` (which is the default and recommended value) and there are at least three master-eligible nodes in the cluster, Elasticsearch remains capable of processing cluster state updates as long as all but one of its master-eligible nodes are healthy.
4140
::::
4241

43-
4442
There are situations in which Elasticsearch might tolerate the loss of multiple nodes, but this is not guaranteed under all sequences of failures. If the `cluster.auto_shrink_voting_configuration` setting is `false`, you must remove departed nodes from the voting configuration manually. Use the [voting exclusions API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-post-voting-config-exclusions) to achieve the desired level of resilience.
4543

4644
No matter how it is configured, Elasticsearch will not suffer from a "split-brain" inconsistency. The `cluster.auto_shrink_voting_configuration` setting affects only its availability in the event of the failure of some of its nodes and the administrative tasks that must be performed as nodes join and leave the cluster.
4745

48-
4946
## Even numbers of master-eligible nodes [_even_numbers_of_master_eligible_nodes]
5047

5148
There should normally be an odd number of master-eligible nodes in a cluster. If there is an even number, Elasticsearch leaves one of them out of the voting configuration to ensure that it has an odd size. This omission does not decrease the failure-tolerance of the cluster. In fact, improves it slightly: if the cluster suffers from a network partition that divides it into two equally-sized halves then one of the halves will contain a majority of the voting configuration and will be able to keep operating. If all of the votes from master-eligible nodes were counted, neither side would contain a strict majority of the nodes and so the cluster would not be able to make any progress.
5249

5350
For instance if there are four master-eligible nodes in the cluster and the voting configuration contained all of them, any quorum-based decision would require votes from at least three of them. This situation means that the cluster can tolerate the loss of only a single master-eligible node. If this cluster were split into two equal halves, neither half would contain three master-eligible nodes and the cluster would not be able to make any progress. If the voting configuration contains only three of the four master-eligible nodes, however, the cluster is still only fully tolerant to the loss of one node, but quorum-based decisions require votes from two of the three voting nodes. In the event of an even split, one half will contain two of the three voting nodes so that half will remain available.
5451

55-
5652
## Setting the initial voting configuration [_setting_the_initial_voting_configuration]
5753

5854
When a brand-new cluster starts up for the first time, it must elect its first master node. To do this election, it needs to know the set of master-eligible nodes whose votes should count. This initial voting configuration is known as the *bootstrap configuration* and is set in the [cluster bootstrapping process](modules-discovery-bootstrap-cluster.md).
@@ -65,5 +61,4 @@ If the bootstrap configuration is not set correctly, when you start a brand-new
6561
To illustrate the problem with configuring each node to expect a certain cluster size, imagine starting up a three-node cluster in which each node knows that it is going to be part of a three-node cluster. A majority of three nodes is two, so normally the first two nodes to discover each other form a cluster and the third node joins them a short time later. However, imagine that four nodes were erroneously started instead of three. In this case, there are enough nodes to form two separate clusters. Of course if each node is started manually then it’s unlikely that too many nodes are started. If you’re using an automated orchestrator, however, it’s certainly possible to get into this situation-- particularly if the orchestrator is not resilient to failures such as network partitions.
6662
::::
6763

68-
6964
The initial quorum is only required the very first time a whole cluster starts up. New nodes joining an established cluster can safely obtain all the information they need from the elected master. Nodes that have previously been part of a cluster will have stored to disk all the information that is required when they restart.

deploy-manage/distributed-architecture/shard-allocation-relocation-recovery/index-level-shard-allocation.md

Lines changed: 20 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -3,31 +3,12 @@ mapped_urls:
33
- https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-allocation.html
44
- https://www.elastic.co/guide/en/elasticsearch/reference/current/shard-allocation-filtering.html
55
- https://www.elastic.co/guide/en/elasticsearch/reference/current/recovery-prioritization.html
6+
applies_to:
7+
stack:
68
---
79

810
# Index-level shard allocation
911

10-
% What needs to be done: Refine
11-
12-
% Scope notes: also reference https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-total-shards.html and https://www.elastic.co/guide/en/elasticsearch/reference/current/data-tier-shard-filtering.html
13-
14-
% Use migrated content from existing pages that map to this page:
15-
16-
% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/index-modules-allocation.md
17-
% Notes: conceptual content
18-
% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/shard-allocation-filtering.md
19-
% Notes: conceptual content
20-
% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/recovery-prioritization.md
21-
% Notes: conceptual content
22-
23-
⚠️ **This page is a work in progress.** ⚠️
24-
25-
The documentation team is working to combine content pulled from the following pages:
26-
27-
* [/raw-migrated-files/elasticsearch/elasticsearch-reference/index-modules-allocation.md](../../../raw-migrated-files/elasticsearch/elasticsearch-reference/index-modules-allocation.md)
28-
* [/raw-migrated-files/elasticsearch/elasticsearch-reference/shard-allocation-filtering.md](../../../raw-migrated-files/elasticsearch/elasticsearch-reference/shard-allocation-filtering.md)
29-
* [/raw-migrated-files/elasticsearch/elasticsearch-reference/recovery-prioritization.md](../../../raw-migrated-files/elasticsearch/elasticsearch-reference/recovery-prioritization.md)
30-
3112
In Elasticsearch, per-index settings allow you to control the allocation of shards to nodes through index-level shard allocation settings. These settings enable you to specify preferences or constraints for where shards of a particular index should reside. This includes allocating shards to nodes with specific attributes or avoiding certain nodes. This level of control helps optimize resource utilization, balance load, and ensure data redundancy and availability according to your deployment's specific requirements. In addition to the content in this article, there are additional resources:
3213

3314
* [Shard allocation filtering](../../../deploy-manage/distributed-architecture/shard-allocation-relocation-recovery/index-level-shard-allocation.md): Controlling which shards are allocated to which nodes.
@@ -45,8 +26,7 @@ The `cluster.routing.allocation` settings are dynamic, enabling existing indices
4526

4627
For example, you could use a custom node attribute to indicate a node’s performance characteristics and use shard allocation filtering to route shards for a particular index to the most appropriate class of hardware.
4728

48-
49-
### Enabling index-level shard allocation filtering [index-allocation-filters]
29+
### Enabling index-level shard allocation filtering [index-allocation-filters]
5030

5131
To filter based on a custom node attribute:
5232

@@ -89,39 +69,25 @@ To filter based on a custom node attribute:
8969

9070
### Index allocation filter settings [index-allocation-settings]
9171

92-
`index.routing.allocation.include.{{attribute}}`
93-
: Assign the index to a node whose `{{attribute}}` has at least one of the comma-separated values.
94-
95-
`index.routing.allocation.require.{{attribute}}`
96-
: Assign the index to a node whose `{{attribute}}` has *all* of the comma-separated values.
97-
98-
`index.routing.allocation.exclude.{{attribute}}`
99-
: Assign the index to a node whose `{{attribute}}` has *none* of the comma-separated values.
72+
| Setting | Description |
73+
|---|---|
74+
|`index.routing.allocation.include.{{attribute}}`| Assign the index to a node whose `{{attribute}}` has at least one of the comma-separated values.|
75+
|`index.routing.allocation.require.{{attribute}}`| Assign the index to a node whose `{{attribute}}` has *all* of the comma-separated values.|
76+
|`index.routing.allocation.exclude.{{attribute}}`| Assign the index to a node whose `{{attribute}}` has *none* of the comma-separated values. |
10077

10178
The index allocation settings support the following built-in attributes:
10279

103-
`_name`
104-
: Match nodes by node name
105-
106-
`_host_ip`
107-
: Match nodes by host IP address (IP associated with hostname)
108-
109-
`_publish_ip`
110-
: Match nodes by publish IP address
111-
112-
`_ip`
113-
: Match either `_host_ip` or `_publish_ip`
114-
115-
`_host`
116-
: Match nodes by hostname
117-
118-
`_id`
119-
: Match nodes by node id
120-
121-
`_tier`
122-
: Match nodes by the node’s [data tier](../../../manage-data/lifecycle/data-tiers.md) role. For more details see [data tier allocation filtering](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/index-settings/data-tier-allocation-settings.md)
123-
124-
::::{note}
80+
| Attribute | Description|
81+
| --- | --- |
82+
|`_name`| Match nodes by node name |
83+
|`_host_ip`| Match nodes by host IP address (IP associated with hostname) |
84+
|`_publish_ip`| Match nodes by publish IP address |
85+
|`_ip`| Match either `_host_ip` or `_publish_ip` |
86+
| `_host`| Match nodes by hostname |
87+
|`_id`| Match nodes by node id |
88+
|`_tier`| Match nodes by the node’s [data tier](../../../manage-data/lifecycle/data-tiers.md) role. For more details see [data tier allocation filtering](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/index-settings/data-tier-allocation-settings.md) |
89+
90+
::::{note}
12591
`_tier` filtering is based on [node](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/node-settings.md) roles. Only a subset of roles are [data tier](../../../manage-data/lifecycle/data-tiers.md) roles, and the generic [data role](../../../deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#data-node-role) will match any tier filtering.
12692
::::
12793

@@ -180,4 +146,4 @@ PUT index_4/_settings
180146
{
181147
"index.priority": 1
182148
}
183-
```
149+
```

0 commit comments

Comments
 (0)