Skip to content

docs: add 429 semantic_text troubleshooting #2507

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Aug 12, 2025
Merged

docs: add 429 semantic_text troubleshooting #2507

Changes from 7 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
767e30f
docs: add 429 semantic_text troubleshooting
mromaios Aug 11, 2025
0fb79ed
adjust text
mromaios Aug 11, 2025
88d61fb
format
mromaios Aug 11, 2025
0206b4c
Update troubleshoot/elasticsearch/rejected-requests.md
mromaios Aug 11, 2025
3769cd7
Merge branch 'add_429_semantic_text_note' of github.com:mromaios/docs…
mromaios Aug 11, 2025
951e2b4
trying out markdown applies
mromaios Aug 11, 2025
e0b1d08
further markdown exp
mromaios Aug 11, 2025
97d4cb2
Update troubleshoot/elasticsearch/rejected-requests.md
mromaios Aug 12, 2025
241c1fa
fix formatting, address pr comments
mromaios Aug 12, 2025
55aa0c4
Merge branch 'add_429_semantic_text_note' of github.com:mromaios/docs…
mromaios Aug 12, 2025
54f8a59
Merge branch 'main' into add_429_semantic_text_note
mromaios Aug 12, 2025
0cb9522
Merge branch 'main' into add_429_semantic_text_note
mromaios Aug 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions troubleshoot/elasticsearch/rejected-requests.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,13 +67,27 @@ These stats are cumulative from node startup.

Indexing pressure rejections appear as an `EsRejectedExecutionException`, and indicate that they were rejected due to `combined_coordinating_and_primary`, `coordinating`, `primary`, or `replica`.

These errors are often related to [backlogged tasks](task-queue-backlog.md), [bulk index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) sizing, or the ingest target's [`refresh_interval` setting](elasticsearch://reference/elasticsearch/index-settings/index-modules.md).
These errors are often related to [backlogged tasks](task-queue-backlog.md), [bulk index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) sizing, or the ingest target's [`refresh_interval` setting](elasticsearch://reference/elasticsearch/index-settings/index-modules.md).
{applies_to}`stack: ga 9.1`{applies_to}`serverless: ga`Another cause of indexing pressure rejections might be the use of the [`semantic_text`](https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/semantic-text) field type, which can cause rejections when indexing large batches of documents if the batch may otherwise incur an Out Of Memory (OOM) error.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line renders oddly due to the the inline applies_to:

Screenshot 2025-08-11 at 1 30 27 PM

Perhaps we should make a new sub-section for semantic text here and use a section applies_to instead? Or put this content in a note block?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried out a note block.
Screenshot 2025-08-12 at 14 10 22
Any better? 😅


See [this video](https://www.youtube.com/watch?v=QuV8QqSfc0c) for a walkthrough of diagnosing indexing pressure rejections.


## Prevent rejected requests [prevent-rejected-requests]

**Fix high CPU and memory usage**
### Fix high CPU and memory usage [fix-high-cpu-memory-usage]

If {{es}} regularly rejects requests and other tasks, your cluster likely has high CPU usage or high JVM memory pressure. For tips, see [High CPU usage](high-cpu-usage.md) and [High JVM memory pressure](high-jvm-memory-pressure.md).

### Fix for `semantic_text` ingestion issues [fix-semantic-text-ingestion-issues]
```{applies_to}
stack: ga 9.1
serverless: ga
```

1. Reduce the batch size of documents in your indexing requests.
2. If reducing batch size doesn't resolve the issue, then consider scaling up your machine resources.
{applies_to}`serverless: unavailable`3. A last resort option is to adjust the `indexing_pressure.memory.coordinating.limit` cluster setting. The default value is 10% of the heap. Increasing this limit allows more memory to be used for coordinating operations before rejections occur.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar rendering issue here:

Screenshot 2025-08-11 at 1 30 57 PM

Moving the applies_to to after the "3." could help

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

1. Reduce the batch size of documents in your indexing requests.
2. If reducing batch size doesn't resolve the issue, then consider scaling up your machine resources.
3. {applies_to}`serverless: unavailable` A last resort option is to adjust the `indexing_pressure.memory.coordinating.limit` cluster setting. The default value is 10% of the heap. Increasing this limit allows more memory to be used for coordinating operations before rejections occur.

Will render as:
screen

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh definitely! Thanks both, apologies! Was playing around with the markdown yesterday and left it in an inconsistent state.
Thanks for the suggestion!

❓ Is there a way to render this locally so that I can try a few things out?

Copy link
Contributor

@kilfoyle kilfoyle Aug 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mromaios Certainly! The instructions for running a local docs build are here: https://elastic.github.io/docs-builder/contribute/locally/

You'd need to install the docs-builder tool and then clone the docs-content repo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks @kilfoyle, works great!

::::{warning}
This adjustment should only be considered after exhausting other options, as setting this value too high may risk Out of Memory (OOM) errors in your cluster. A cluster restart is required for this change to take effect.
::::
Loading