Introduce an ignore_above index-level setting
#113121
Conversation
elasticsearchmachine
added
needs:triage
salvatore-campagna
changed the title
ignore_above index-level setting
salvatore-campagna
added
>non-issue
:StorageEngine/Logs
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
| return (KeywordFieldMapper) in; | ||
| } | ||
|
|
||
| public static Builder buildForTest(final String name, boolean hasDocValues) { |
There was a problem hiding this comment.
I don't think this is needed and is unrelated to adding the new index setting? We can keep using the constructor with two parameters?
There was a problem hiding this comment.
I removed this after changing the available constructors.
| match_all: {} | ||
|
|
||
| - length: { hits.hits: 1 } | ||
| #TODO: synthetic source field reconstruction bug (TBD: add link to the issue here) |
There was a problem hiding this comment.
I could not reproduce the issue using only a keyword field...it looks like the issue happens when a keyword is used together with a flattened field.
| public abstract class Mapper implements ToXContentFragment, Iterable<Mapper> { | ||
|
|
||
| public static final NodeFeature SYNTHETIC_SOURCE_KEEP_FEATURE = new NodeFeature("mapper.synthetic_source_keep"); | ||
| public static final NodeFeature IGNORE_ABOVE_INDEX_LEVEL_SETTING = new NodeFeature("mapper.ignore_above_index_level_setting"); |
There was a problem hiding this comment.
maybe move this constant also to IndexSettings?
| */ | ||
| protected abstract boolean modifySearch(ApiCallSection search); | ||
|
|
||
| private static Object getSetting(final Object map, final String... keys) { |
There was a problem hiding this comment.
This change is necessary to skip testing with runtime fields when ignore_above is applied. Runtime fields don't support ignore_above, and the behavior is already in place at the mapping level (see code below). This ensures that tests don't incorrectly use runtime fields in scenarios where ignore_above is used (we skip them).
Below, in method runtimeifyMappingProperties, we do the following indeed, if ignore_above is used at field mapping level:
if (propertyMap.containsKey("ignore_above")) {
// Scripts don't support ignore_above so we skip those fields
continue;
}
b2b4015 to
086cc44
Compare
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
Here we introduce a new index-level setting, `ignore_above`, similar to what we have for `ignore_malformed`. The setting will apply to all `keyword`, `wildcard` and `flattened` fields. Each field mapping will still be allowed to override the index-level setting using a mapping-level `ignore_above` value. (cherry picked from commit 208a1fe)
Here we introduce a new index-level setting, `ignore_above`, similar to what we have for `ignore_malformed`. The setting will apply to all `keyword`, `wildcard` and `flattened` fields. Each field mapping will still be allowed to override the index-level setting using a mapping-level `ignore_above` value. (cherry picked from commit 208a1fe)
5 participants
Here we introduce a new index-level setting,
ignore_above, similar to what we havefor
ignore_malformed. The setting will apply to allkeyword,wildcardandflattenedfields. Each field mapping will still be allowed to override the index-level setting using a
mapping-level
ignore_abovevalue.Resolves #112795