Skip to content

Commit b4c2c2e

Browse files
stefnestorkilfoylenielsbauman
authored
(Doc) ILM Force Merge not on HDD and happens on hosting node not current phase tier (#130280)
* Update ilm-forcemerge.md Co-authored-by: David Kilfoyle <[email protected]> Co-authored-by: Niels Bauman <[email protected]>
1 parent cc62555 commit b4c2c2e

File tree

1 file changed

+11
-10
lines changed

1 file changed

+11
-10
lines changed

docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -7,29 +7,30 @@ mapped_pages:
77

88
Phases allowed: hot, warm.
99

10-
[Force merges](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) the index into the specified maximum number of [segments](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments).
11-
12-
::::{note}
13-
Shards that are relocating during a `forcemerge` will not be merged.
14-
::::
15-
10+
[Force merges](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) the index into the specified maximum number of [segments](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments). This operation is performed on a best effort basis. For example, shards that are relocating during a `forcemerge` will not be merged.
1611

1712
To use the `forcemerge` action in the `hot` phase, the `rollover` action **must** be present. If no rollover action is configured, {{ilm-init}} will reject the policy.
1813

1914
:::::{admonition} Performance considerations
2015
:name: ilm-forcemerge-performance
2116

22-
Force merge is a resource-intensive operation. If too many force merges are triggered at once, it can negatively impact your cluster. This can happen when you apply an {{ilm-init}} policy that includes a force merge action to existing indices. If they meet the `min_age` criteria, they can immediately proceed through multiple phases. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated.
17+
Force merge is a resource-intensive operation. If too many force merges are triggered at once, it can negatively impact your cluster. For example, this can happen when you
18+
* modify an existing {{ilm-init}} policy's phase `min_age`, causing indices to trigger the force merge at a faster rate.
19+
* apply an {{ilm-init}} policy that includes a force merge action to existing indices. If the indices meet the `min_age` criteria, they can immediately proceed through multiple actions. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated.
20+
* run the [{{ilm-init}} Move Step API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-move-to-step) against multiple indices.
2321

2422
If you experience a force merge task queue backlog, you might need to increase the size of the force merge threadpool so indices can be force merged in parallel. To do this, configure the `thread_pool.force_merge.size` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings).
2523

2624
::::{important}
27-
This can have cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog.
25+
Note that `thread_pool.force_merge.size` is an advanced setting. Adjusting it can cause cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog.
2826
::::
2927

28+
Force merging will be performed by the node hosting the shard. Usually, the [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}} phase that the index is in. One of the exceptions is when you have manually disabled [ILM Migrate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-migrate) and have specified custom allocations using [ILM allocate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-allocate). The other exception is searchable snapshots; force merges for [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` are performed in the phase that the index is in **prior** to the `searchable_snapshot` action. You may want to explicitly choose in which data tier the force merge should occur, for example:
29+
* A force merge in the `hot` phase will use hot nodes. Merges may be faster on this potentially higher performance hardware but may have the tradeoff of impacting ingestion.
30+
* A force merge in the `warm` phase will use warm nodes. Merges may take longer to perform on potentially lower performance hardware but will avoid impacting ingestion in the `hot` tier.
31+
* [{{ilm-init}} Searchable Snapshot](./ilm-searchable-snapshot.md) performance is dependant upon the shard having been force merged, so by default this ILM action will enable `force_merge_index`. This will trigger force merges in the preceding node data tier for `cold` and `frozen` phases.
3032

31-
Force merging will be performed by the nodes within the current phase of the index. A forcemerge in the `hot` phase will use hot nodes with potentially faster nodes, while impacting ingestion more. A forcemerge in the `warm` phase will use warm nodes and potentially take longer to perform, but without impacting ingestion in the `hot` tier.
32-
33+
We recommend that merges be targetted against SSD and not HDD disks.
3334
:::::
3435

3536

0 commit comments

Comments
 (0)