-
Notifications
You must be signed in to change notification settings - Fork 159
docs: add 429 semantic_text troubleshooting #2507
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
767e30f
0fb79ed
88d61fb
0206b4c
3769cd7
951e2b4
e0b1d08
97d4cb2
241c1fa
55aa0c4
54f8a59
0cb9522
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -67,7 +67,9 @@ These stats are cumulative from node startup. | |
|
||
Indexing pressure rejections appear as an `EsRejectedExecutionException`, and indicate that they were rejected due to `combined_coordinating_and_primary`, `coordinating`, `primary`, or `replica`. | ||
|
||
These errors are often related to [backlogged tasks](task-queue-backlog.md), [bulk index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) sizing, or the ingest target's [`refresh_interval` setting](elasticsearch://reference/elasticsearch/index-settings/index-modules.md). | ||
These errors are often related to [backlogged tasks](task-queue-backlog.md), [bulk index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) sizing, or the ingest target's [`refresh_interval` setting](elasticsearch://reference/elasticsearch/index-settings/index-modules.md).¹ | ||
|
||
¹ : Starting from versions 8.19+ and 9.1+, indexing to a semantic_text field can also cause 429 errors if the batch may otherwise incur an Out Of Memory (OOM) error. | ||
|
||
See [this video](https://www.youtube.com/watch?v=QuV8QqSfc0c) for a walkthrough of diagnosing indexing pressure rejections. | ||
|
||
|
@@ -77,3 +79,11 @@ See [this video](https://www.youtube.com/watch?v=QuV8QqSfc0c) for a walkthrough | |
**Fix high CPU and memory usage** | ||
|
||
If {{es}} regularly rejects requests and other tasks, your cluster likely has high CPU usage or high JVM memory pressure. For tips, see [High CPU usage](high-cpu-usage.md) and [High JVM memory pressure](high-jvm-memory-pressure.md). | ||
|
||
**Fix for semantic_text field issues:** | ||
|
||
When working with semantic_text fields in versions 8.19+ and 9.1+: | ||
mromaios marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
mromaios marked this conversation as resolved.
Show resolved
Hide resolved
|
||
1. First, try reducing the batch size of documents in your indexing requests. | ||
mromaios marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
2. If reducing batch size doesn't resolve the issue, then consider scaling up your machine resources. | ||
3. A last resort option is to adjust the indexing_pressure.memory.coordinating.limit cluster setting. The default value is 10% of the heap. Increasing this limit allows more memory to be used for coordinating operations before rejections occur. This adjustment should only be considered after exhausting other options, as setting this value too high may risk Out of Memory (OOM) errors in your cluster. A cluster restart is required for this change to take effect. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we typically use footnotes in our docs? It would probably be better to put this in the doc and use our applies to syntax to tag the appropriate version number. Note that we'll want a follow up PR for 8.19 as this uses asciidoc format, so we'll only need applies to 9.1 in markdown.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes perfect sense, (I just like footnotes 🙈). Will adjust, thanks!