-
Notifications
You must be signed in to change notification settings - Fork 195
Enhance mapping explosion documentation in Elasticsearch #4539
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Expanded explanation of mapping explosion in Elasticsearch, including its causes, symptoms, and how to check for issues. - Added details on the default field limit and its implications for performance. - Replaced the intro to avoid the curse of knowledge (intro assumed customer's knew what mappings and field types were).
Vale Linting ResultsSummary: 4 suggestions found 💡 Suggestions (4)
|
🔍 Preview links for changed docs |
yetanothertw
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for making these changes, they look great!
I've left a couple of small style suggestions for your consideration.
Co-authored-by: Vlada Chirmicci <[email protected]>
Co-authored-by: Vlada Chirmicci <[email protected]>
Co-authored-by: Vlada Chirmicci <[email protected]>
yetanothertw
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thank you! 🪴
| To confirm the field totals of an index to check for mapping explosion: | ||
|
|
||
| * Check {{es}} cluster logs for errors `Limit of total fields [X] in index [Y] has been exceeded` where `X` is the value of `index.mapping.total_fields.limit` and `Y` is your index. The correlated ingesting source log error would be `Limit of total fields [X] has been exceeded while adding new fields [Z]` where `Z` is attempted new fields. | ||
| * Check {{es}} cluster logs for errors similar to `Limit of total fields [X] in index [Y] has been exceeded` where `X` is the value of `index.mapping.total_fields.limit` and `Y` is your index. The correlated ingesting source log ({{beats}}, {{agent}}, or {{ls}} logs) error would be `Limit of total fields [X] has been exceeded while adding new fields [Z]` where `Z` is attempted new fields. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- IMO calling out Agent where Elastic-mainly controls the pipeline/integration suggests poor form of us pushing a product issue onto them.
- We frequently use ingesting
client-sideforthe correlated ingesting source log. - Also worth calling out here/above that this error also contains HTTP
400andmapper_parsing_exceptionwhich is what users more commonly google.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- My apologies if that was the read. I was not intending to push a product issue on to the customer. Was trying to better describe where you would see the error.
- The intent was to call out what the error would look like as the original "correlated ingesting source log" reference seemed a bit vague i.e. could we be saying their Kafka, in-house application, code client libraries, etc. would have that error message? So thought calling it out implicitly what products would have that message. Is there an in-house writer's bible I could read up on more of these nomenclatures, like
client-side?
EDIT: doh meant explicitly instead of implicitly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Totally no worries! Sorry, not trying to call you out vs I think our KBs do that widespread 🙂
- I think the team's currently revamping this doc but am unsure where this glossary would be so was just FYI
- (leaving placeholder)
| * Check {{es}} cluster logs for errors similar to `Limit of total fields [X] in index [Y] has been exceeded` where `X` is the value of `index.mapping.total_fields.limit` and `Y` is your index. The correlated ingesting source log ({{beats}}, {{agent}}, or {{ls}} logs) error would be `Limit of total fields [X] has been exceeded while adding new fields [Z]` where `Z` is attempted new fields. | |
| * Check {{es}} cluster logs for errors similar to `Limit of total fields [X] in index [Y] has been exceeded` where `X` is the value of `index.mapping.total_fields.limit` and `Y` is the index name. The correlated client-side ingesting source HTTP 400 `mapper_parsing_exception` log error would be `Limit of total fields [X] has been exceeded while adding new fields [Z]` where `Z` is attempted new fields. |
Co-authored-by: Stef Nestor <[email protected]>
Co-authored-by: Stef Nestor <[email protected]>
Co-authored-by: Stef Nestor <[email protected]>
Summary
As part of @stefnestor 2026 Support KB alignment project took the latest information on mapping explosions and filled in to our existing troubleshooting documentation.
Generative AI disclosure