Skip to content

(Doc) ILM Force Merge not on HDD and happens on hosting node not current phase tier #130280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Aug 1, 2025
Merged

(Doc) ILM Force Merge not on HDD and happens on hosting node not current phase tier #130280

merged 7 commits into from
Aug 1, 2025

Conversation

@stefnestor
Copy link
Contributor

👋 howdy, team!

A handful of changes for the top of the ILM Force Merge doc, please have Dev confirm 🙏

  1. (Main) Elasticsearch Dev now recommends merges target SSD-not-HDD. (Part of the cultural shift of warm tier having HDD to SSD disk, but specifically merges are much faster on SSD.)
  2. Force Merges aren't guaranteed to happen on the phase's data tier ever since Searchable Snapshots have existed. Related info, video. Ex: you can force merge on cold for cold>frozen even though force merge action not allowed in cold & you never force merge on frozen.
  3. added some "how you can accidentally kick this off"

I'm not tied to formatting at all. And kindly fix links as I'm catching up to the new system. However it needs said, kindly help ensure (1) and (2) correct but feel free to revert (3) as desired 🙏 .

@stefnestor
Update ilm-forcemerge.md
3b66fa0 
👋 howdy, team!

A handful of changes
@stefnestor stefnestor added >bug >docs General docs changes Team:Docs Meta label for docs team labels Jun 28, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jun 28, 2025 •
edited
Loading

🔍 Preview links for changed docs:

🔔 The preview site may take up to 3 minutes to finish building. These links will become live once it completes.

@elasticsearchmachine
Copy link
Collaborator

@stefnestor please enable the option "Allow edits and access to secrets by maintainers" on your PR. For more information, see the documentation.

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-docs (Team:Docs)

@elasticsearchmachine elasticsearchmachine added v9.2.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Jun 28, 2025
@kilfoyle
Copy link
Contributor

kilfoyle commented Jun 30, 2025 •
edited
Loading

Looks good @stefnestor!

I've added a few comments and have tagged the Data Management team about your questions in the PR description.

@kilfoyle kilfoyle requested a review from a team June 30, 2025 18:19
@stefnestor @kilfoyle
grammar 💚
8edeb7f 
Co-authored-by: David Kilfoyle <[email protected]>
@stefnestor @kilfoyle
feedback 🙏
194cabb 
Co-authored-by: David Kilfoyle <[email protected]>
kilfoyle
kilfoyle approved these changes Jun 30, 2025
View reviewed changes
Copy link
Contributor

@kilfoyle kilfoyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀
Thanks Stef!

nielsbauman
nielsbauman reviewed Jul 1, 2025
View reviewed changes
Copy link
Contributor

@nielsbauman nielsbauman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @stefnestor, thanks for working on this! I left a few first relatively minor comments.

docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md Outdated
Note that `thread_pool.force_merge.size` is an advanced setting. Adjusting it can cause cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog.
::::

Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, you may choose to adjust this behavior. For example:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The node's role frequently matches the data tier of the ILM's phase of the index, you may choose to adjust this behavior.

I'm having trouble understanding this sentence, but maybe it's just me. Could you explain what you mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah @kilfoyle and I tried to has this out but I'm not sure I got it across the line, sorry.

The needed fix from current doc: force merges do not happen on the ilm phase's node tier. They happen on the node hosting the shard, this specifically is different when you use searchable snapshots (e.g. hot>frozen merges on hot inside phase/action/step: frozen/searchable_snapshot/forcemerge).

I've debated just removing you may choose to adjust this behavior. David & I discussed it as relating to disabling ILM Migrate (and using node attribute allocation filters).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks for the explanation, that helps a lot! How about something like this:

Suggested change
Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, you may choose to adjust this behavior. For example:
Force merging will be performed by the node hosting the shard. Usually, the [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}} phase that the index is in. One of the exceptions is when you have manually disabled [ILM Migrate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-migrate) and have specified custom allocations using [ILM allocate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-allocate). The other exception is searchable snapshots; force merges for [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` are performed in the phase that the index is in **prior** to the `searchable_snapshot` action. You may want to explicitly choose in which data tier the force merge should occur, for example:

in combination with your suggestion in the other thread below.
Optionally, we could link to the force_merge_index option on https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-searchable-snapshot#ilm-searchable-snapshot-options, for more details on force merging for searchable snapshots.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I forgot to mention that we should fix those links that I added. Do you happen to know what the correct links are? I'm not familiar with the new docs system yet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

David usually fixes them for me as I'm also getting up to speed on the new system 🙏

@stefnestor @nielsbauman
feedback 🙏
878e9f7 
Co-authored-by: Niels Bauman <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Jul 2, 2025 •
edited
Loading

🔍 Preview links for changed docs

@stefnestor @nielsbauman
feedback 🙏
0ee8c3f 
Co-authored-by: Niels Bauman <[email protected]>
@nielsbauman
Copy link
Contributor

@kilfoyle could you have one final look as well, please? 🙏

@stefnestor stefnestor merged commit b4c2c2e into main Aug 1, 2025
10 checks passed
@stefnestor stefnestor deleted the stefnestor-patch-3 branch August 1, 2025 17:14
szybia added a commit to szybia/elasticsearch that referenced this pull request Aug 1, 2025
@szybia
Merge remote-tracking branch 'upstream/main' into index-templates-tra…
08aec70 
…cking

* upstream/main: (166 commits)
  Reduce inactive sink interval in VectorSimilarityFunctionsIT (elastic#132288)
  ESQL: Allow agg tests to process many columns (elastic#132358)
  Update analysis-lowercase-tokenfilter.md (elastic#132359)
  Add Sparse Vector Index Options Settings to Semantic Text Field (elastic#131058)
  Collect node thread pool usage for shard balancing (elastic#131480)
  Add tasks to validate new style transport versions (elastic#131782)
  Mute org.elasticsearch.search.routing.SearchReplicaSelectionIT testNodeSelection elastic#132354
  Mute org.elasticsearch.xpack.esql.action.CrossClusterAsyncQueryIT testBadAsyncId elastic#132353
  Fixes DenseVectorFieldIndexTypeUpdateIT release tests (elastic#132346)
  Fix testCloseOrReallocateDuringPartialSnapshot (elastic#132049)
  (Doc) ILM Force Merge not on HDD and happens on hosting node not current phase tier (elastic#130280)
  Run GeoIp YAML tests in multi-project cluster and fix bug discovered by tests (elastic#131521)
  Unmutes elastic#132111, seems a transient, non reproducible issue (elastic#132253)
  Mute org.elasticsearch.search.suggest.phrase.PhraseSuggesterIT testPhraseSuggestionWithNgramOnlyAnalyzerThrowsException elastic#132347
  Add AI21 support to Inference Plugin (elastic#131238)
  OpenJDK EA builds should use https instead of http (elastic#132297)
  ESQL: Normalize timeseries aggs slightly (elastic#132284)
  Avoid internal server error on suggester ngram bad request (elastic#132321)
  [ES|QL] Rerank operator improvements (elastic#132318)
  Mute org.elasticsearch.xpack.logsdb.qa.LogsDbVersusReindexedLogsDbChallengeRestIT testTermsQuery elastic#132337
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nielsbauman nielsbauman nielsbauman left review comments

@kilfoyle kilfoyle kilfoyle approved these changes

Assignees

No one assigned

Labels

>bug >docs General docs changes external-contributor Pull request authored by a developer outside the Elasticsearch team Team:Docs Meta label for docs team v9.2.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@stefnestor @elasticsearchmachine @kilfoyle @nielsbauman