Doc values sparse index on _tsid fields #122699
Conversation
jordan-powers
changed the title
jordan-powers
force-pushed
the
tsid-sparse-doc-values
branch
from
5412486 to
1b210b0
Compare
jordan-powers
force-pushed
the
tsid-sparse-doc-values
branch
from
1b210b0 to
25749f4
Compare
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
| context.doc().add(new SortedDocValuesField(fieldType().name(), timeSeriesId)); | ||
|
|
||
| if (this.useDocValuesSkipper && this.indexCreatedVersion.onOrAfter(IndexVersions.TIME_SERIES_ID_DOC_VALUES_SPARSE_INDEX)) { | ||
| context.doc().add(SortedDocValuesField.indexedField(fieldType().name(), timeSeriesId)); |
There was a problem hiding this comment.
SortedDocValuesField#indexedField creates a new SortedDocValuesField using a private static FieldType that has the skip index enabled. It's the same as discussed for SortedNumericDocValuesField in #122161 (comment)
| public static final String NAME = "_tsid"; | ||
| public static final String CONTENT_TYPE = "_tsid"; | ||
| public static final TimeSeriesIdFieldType FIELD_TYPE = new TimeSeriesIdFieldType(); | ||
| public static final TimeSeriesIdFieldMapper INSTANCE = new TimeSeriesIdFieldMapper(); |
There was a problem hiding this comment.
The reason why this is a singleton constant is to reduce memory usage of MapperService. Meta field mappers that can be initialized as a constant are a constant. A node can have many MapperService instances (one per allocated index).
I think can just define two singleton instances here and then decide to use which in IndexMode#timeSeriesIdFieldMapper(...) based on index versions and whether the feature flag is enabled?
|
|
||
| private TimeSeriesIdFieldMapper() { | ||
| private final IndexVersion indexCreatedVersion; | ||
| private final boolean useDocValuesSkipper; |
There was a problem hiding this comment.
I think we can avoid passing down index version here. We can set useDocValuesSkipper based on the following expressions: indexVersion.onOrAfter(IndexVersions.TIME_SERIES_ID_DOC_VALUES_SPARSE_INDEX) && context.getIndexSettings().useDocValuesSkipper()?
3 participants
This PR exends the work done in #122161 and #121751 to also use the doc values sparse index for the _tsid fields in time-series mode indices.