Set default_metric for time series on the index level #133464

limotova · 2025-08-25T08:50:31Z

This PR adds in an index-level setting that allows you to specify which metric (min, max, sum, value_count) will be used as the default metric when downsampling indices containing gauges in time series indices.
When the setting is not present, it defaults to max.
If an aggregate_metric_double has a setting different from the index-level setting, the aggregate_metric_double's setting takes precedence for that field.

Example settings/mappings:

"settings": {
  "index.mode": "time_series",
  "index.time_series.default_metric": "min",
  ...
},
"mappings": {
  "properties": {
    "my-metric": {
      "type": "double",
      "time_series_metric": "gauge"
    },
  ...
  }
}

becomes this after downsampling:

"my-metric": {
    "type": "aggregate_metric_double",
    "metrics": [
        "min",
        "max",
        "sum",
        "value_count"
    ],
    "default_metric": "min",
    "time_series_metric": "gauge"
}

limotova · 2025-08-26T03:40:42Z

Alternative approach to #132626

jordan-powers

I had a couple of questions/nits, but overall LGTM!

jordan-powers · 2025-08-26T21:05:01Z

...n/downsample/src/main/java/org/elasticsearch/xpack/downsample/TransportDownsampleAction.java

@@ -783,13 +785,19 @@ public static AggregateMetricDoubleFieldSupportedMetrics getSupportedMetrics(
                (String) fieldProperties.get(AggregateMetricDoubleFieldMapper.Names.DEFAULT_METRIC),
                defaultMetric
            );
+        } else {
+            defaultMetric = indexLevelDefaultMetric.name().toLowerCase(Locale.ROOT);


If I'm reading this correctly, index.time_series.default_metric will only apply to non-aggregate number fields that are being downsampled into aggregate metrics, and existing aggregate metrics with no specified default_metric will still fall back to "max".

Would it makes sense to instead have index.time_series.default_metric specify the default metric in both cases?

I think this does make sense. Maybe the defaultMetric variable should just be initialized by indexLevelDefaultMetric (on line 777 in main branch)? Given that the default of the new index setting is also max.

I think I'm not entirely sure I understand the situation
I had an assumption that if you don't specify the default metric for an aggregate metric double field, then it's automatically assigned one (which would be max if max is present), which would mean all existing aggregate metric doubles have a default metric - is that not the case?
I was thinking I want to avoid a situation where someone already has an aggregate metric double, and then this setting gets introduced and it overwrites their old aggregate metric double's default setting

I think you're right--in most cases, there will already be a default metric specified on the aggregate metric, since aggregate metrics are usually created by downsampling which adds a default metric using this logic.

But it is technically possible to manually create an aggregate metric field that has no default metric specified. In this case, the logic here is hard-coded to default to using "max" as the default metric. But it might make more sense to instead use the new index-level index.time_series.default_metric here.

jordan-powers · 2025-08-26T21:08:16Z

server/src/main/java/org/elasticsearch/index/IndexSettings.java

+    public enum AggregateMetricDoubleDefaultMetric {
+        MIN,
+        MAX,
+        SUM,
+        VALUE_COUNT;
+    }


nit: It feels a bit smelly to have this enum duplicate the other enum AggregateMetricDoubleFieldMapper.Metric. It makes sense since that second enum only exists in the mapper-aggregate-metric plug-in and so can't be referenced here, but I wonder if you could instead create one enum somewhere and have both classes refer to that one enum?

Yeah we have a few of these enums....there's similarly one in the ES|QL side too and I would love to have just one enum but I'm not sure where it would live

martijnvg

Thanks Larisa, this direction looks good to me. My main comment is around try to move the new setting to the downsample plugin module.

martijnvg · 2025-08-27T07:11:30Z

server/src/main/java/org/elasticsearch/index/IndexSettings.java

+        VALUE_COUNT;
+    }
+
+    public static final Setting<AggregateMetricDoubleDefaultMetric> TIME_SERIES_DEFAULT_METRIC = Setting.enumSetting(


Can this setting be moved to the xpack downsample module? We can just parse it in TimeseriesFieldTypeHelper and keep it around as a field there? (The setting can then be registered in Downsample plugin class by overwriting the getSettings() method)

If that works then there is no need to define the enum that Jordan added a comment about, given that I think we can just use AggregateMetricDoubleFieldMapper.Metric directly in the downsample module (seems to have a compile dependency on mapper-aggregate-metric module).

I will try it out..!

martijnvg · 2025-08-27T07:14:35Z

modules/data-streams/src/main/java/org/elasticsearch/datastreams/DataStreamFeatures.java

@@ -30,6 +30,10 @@ public class DataStreamFeatures implements FeatureSpecification {

    public static final NodeFeature FAILURE_STORE_IN_LOG_DATA_STREAMS = new NodeFeature("logs_data_streams.failure_store.enabled");

+    public static final NodeFeature DOWNSAMPLE_INDEX_LEVEL_DEFAULT_METRIC = new NodeFeature(


Given that this is a downsample feature. I think we should add this feature to the downsample module? It doesn't seem to have a class extending FeatureSpecification yet, but I think we can add that.

martijnvg · 2025-08-27T07:18:44Z

...n/downsample/src/main/java/org/elasticsearch/xpack/downsample/TransportDownsampleAction.java

@@ -783,13 +785,19 @@ public static AggregateMetricDoubleFieldSupportedMetrics getSupportedMetrics(
                (String) fieldProperties.get(AggregateMetricDoubleFieldMapper.Names.DEFAULT_METRIC),
                defaultMetric
            );
+        } else {
+            defaultMetric = indexLevelDefaultMetric.name().toLowerCase(Locale.ROOT);


I think this does make sense. Maybe the defaultMetric variable should just be initialized by indexLevelDefaultMetric (on line 777 in main branch)? Given that the default of the new index setting is also max.

elasticsearchmachine added the v9.2.0 label Aug 25, 2025

limotova force-pushed the default-metric-index-level branch 5 times, most recently from 1ac8ce9 to 5a22bb2 Compare August 26, 2025 01:06

Set default_metric on index-level

c871bbd

limotova force-pushed the default-metric-index-level branch from 5a22bb2 to c871bbd Compare August 26, 2025 02:28

limotova requested review from martijnvg, jordan-powers and kkrik-es August 26, 2025 03:39

limotova changed the title ~~Set default_metric on index-level~~ Set default_metric for time series on the index-level Aug 26, 2025

limotova changed the title ~~Set default_metric for time series on the index-level~~ Set default_metric for time series on the index level Aug 26, 2025

jordan-powers approved these changes Aug 26, 2025

View reviewed changes

martijnvg reviewed Aug 27, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Set default_metric for time series on the index level #133464

Set default_metric for time series on the index level #133464

limotova commented Aug 25, 2025 •

edited

Loading

Uh oh!

limotova commented Aug 26, 2025

Uh oh!

jordan-powers left a comment

Uh oh!

jordan-powers Aug 26, 2025

Uh oh!

martijnvg Aug 27, 2025

Uh oh!

limotova Aug 27, 2025

Uh oh!

jordan-powers Aug 27, 2025

Uh oh!

jordan-powers Aug 26, 2025

Uh oh!

limotova Aug 27, 2025

Uh oh!

martijnvg left a comment

Uh oh!

martijnvg Aug 27, 2025

Uh oh!

limotova Aug 27, 2025

Uh oh!

martijnvg Aug 27, 2025

Uh oh!

martijnvg Aug 27, 2025

Uh oh!

Uh oh!

		@@ -30,6 +30,10 @@ public class DataStreamFeatures implements FeatureSpecification {

		public static final NodeFeature FAILURE_STORE_IN_LOG_DATA_STREAMS = new NodeFeature("logs_data_streams.failure_store.enabled");

		public static final NodeFeature DOWNSAMPLE_INDEX_LEVEL_DEFAULT_METRIC = new NodeFeature(

Set default_metric for time series on the index level #133464

Are you sure you want to change the base?

Set default_metric for time series on the index level #133464

Conversation

limotova commented Aug 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

limotova commented Aug 26, 2025

Uh oh!

jordan-powers left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

martijnvg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

limotova commented Aug 25, 2025 •

edited

Loading