Skip to content

(Doc) ILM Force Merge not on HDD and happens on hosting node not current phase tier #130280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Aug 1, 2025
Merged

(Doc) ILM Force Merge not on HDD and happens on hosting node not current phase tier #130280

Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3b66fa0
Update ilm-forcemerge.md
stefnestor Jun 28, 2025
8edeb7f
grammar 💚
stefnestor Jun 30, 2025
fa317e3
Update docs/reference/elasticsearch/index-lifecycle-actions/ilm-force…
kilfoyle Jun 30, 2025
f7165e7
feedback 🙏
stefnestor Jun 30, 2025
194cabb
feedback 🙏
stefnestor Jun 30, 2025
878e9f7
feedback 🙏
stefnestor Jul 2, 2025
0ee8c3f
feedback 🙏
stefnestor Jul 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,29 +7,30 @@ mapped_pages:

Phases allowed: hot, warm.

[Force merges](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) the index into the specified maximum number of [segments](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments).

::::{note}
Shards that are relocating during a `forcemerge` will not be merged.
::::

[Force merges](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) the index into the specified maximum number of [segments](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments). This operation is performed on a best effort basis. For example, shards that are relocating during a `forcemerge` will not be merged.

To use the `forcemerge` action in the `hot` phase, the `rollover` action **must** be present. If no rollover action is configured, {{ilm-init}} will reject the policy.

:::::{admonition} Performance considerations
:name: ilm-forcemerge-performance

Force merge is a resource-intensive operation. If too many force merges are triggered at once, it can negatively impact your cluster. This can happen when you apply an {{ilm-init}} policy that includes a force merge action to existing indices. If they meet the `min_age` criteria, they can immediately proceed through multiple phases. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated.
Force merge is a resource-intensive operation. If too many force merges are triggered at once, it can negatively impact your cluster. For example, this can happen when you
* modify an existing {{ilm-init}} policy's phase `min_age` such that indices trigger into force-merging phase faster.
* apply an {{ilm-init}} policy that includes a force merge action to existing indices. If the indices meet the `min_age` criteria, they can immediately proceed through multiple actions. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated.
* run the [{{ilm-init}} Move Step API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-move-to-step) against multiple indices.

If you experience a force merge task queue backlog, you might need to increase the size of the force merge threadpool so indices can be force merged in parallel. To do this, configure the `thread_pool.force_merge.size` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings).
If you experience a force merge task queue backlog, you might need to increase the size of the force merge threadpool so indices can be force merged in parallel. To do this, configure the `thread_pool.force_merge.size` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings).

::::{important}
This can have cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog.
Note that `thread_pool.force_merge.size` is an advanced setting. Adjusting it can cause cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog.
::::

Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, you may choose to adjust this behavior. For example:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The node's role frequently matches the data tier of the ILM's phase of the index, you may choose to adjust this behavior.

I'm having trouble understanding this sentence, but maybe it's just me. Could you explain what you mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah @kilfoyle and I tried to has this out but I'm not sure I got it across the line, sorry.

The needed fix from current doc: force merges do not happen on the ilm phase's node tier. They happen on the node hosting the shard, this specifically is different when you use searchable snapshots (e.g. hot>frozen merges on hot inside phase/action/step: frozen/searchable_snapshot/forcemerge).

I've debated just removing you may choose to adjust this behavior. David & I discussed it as relating to disabling ILM Migrate (and using node attribute allocation filters).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks for the explanation, that helps a lot! How about something like this:

Suggested change
Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, you may choose to adjust this behavior. For example:
Force merging will be performed by the node hosting the shard. Usually, the [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}} phase that the index is in. One of the exceptions is when you have manually disabled [ILM Migrate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-migrate) and have specified custom allocations using [ILM allocate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-allocate). The other exception is searchable snapshots; force merges for [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` are performed in the phase that the index is in **prior** to the `searchable_snapshot` action. You may want to explicitly choose in which data tier the force merge should occur, for example:

in combination with your suggestion in the other thread below.
Optionally, we could link to the force_merge_index option on https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-searchable-snapshot#ilm-searchable-snapshot-options, for more details on force merging for searchable snapshots.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I forgot to mention that we should fix those links that I added. Do you happen to know what the correct links are? I'm not familiar with the new docs system yet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

David usually fixes them for me as I'm also getting up to speed on the new system 🙏

* A force merge in the `hot` phase will use hot nodes. Merges may be faster on this potentially higher performance hardware but may have the tradeoff of impacting ingestion.
* A force merge in the `warm` phase will use warm nodes. Merges may take longer to perform on potentially lower performance hardware but will avoid impacting ingestion in the `hot` tier.
* A force merge in the `cold` or `frozen` phase [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` happens on the preceeding data tier.

Force merging will be performed by the nodes within the current phase of the index. A forcemerge in the `hot` phase will use hot nodes with potentially faster nodes, while impacting ingestion more. A forcemerge in the `warm` phase will use warm nodes and potentially take longer to perform, but without impacting ingestion in the `hot` tier.

We recommend that merges be targetted against SSD and not HDD disks.
:::::


Expand Down
Loading