[ES|QL] Enable aggregate_metric_double outside of snapshots #135213

limotova · 2025-09-22T17:26:32Z

This commit enables aggregate_metric_double in ES|QL in non-snapshot builds

Many tests (such as the multi-value tests) were modified in order to handle lack-of-support of aggregate_metric_double and should be addressed in follow-up tasks

github-actions · 2025-09-30T04:44:57Z

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

Check out the cumulative docs guidelines
Reach out in the #docs Slack channel

elasticsearchmachine · 2025-09-30T09:43:09Z

Pinging @elastic/es-storage-engine (Team:StorageEngine)

limotova · 2025-09-30T11:00:07Z

x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test/esql/46_downsample.yml

            FROM test-*
            | EVAL a = TO_AGGREGATE_METRIC_DOUBLE(1)  // Temporary workaround to enable aggregate_metric_double
-            | WHERE k8s.pod.uid == \"947e4ced-1786-4e53-9e0c-5c447e959507\"
+            | WHERE k8s.pod.uid == "947e4ced-1786-4e53-9e0c-5c447e959507"


It turns out this test (Stats from downsampled and non-downsampled index simultaneously with implicit casting) wasn't being run due to a typo in one of the capability names, so I had to make a couple of adjustments to it after changing the capabilities

limotova · 2025-09-30T11:05:00Z

x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test/esql/46_downsample.yml

      esql.query:
        body:
-          query: "TS test-* | STATS max = max(k8s.pod.network.rx)  | LIMIT 100"
+          query: "FROM test-* | STATS max = max(k8s.pod.network.rx)  | LIMIT 100"


Same test as above comment (Stats from downsampled and non-downsampled index simultaneously with implicit casting), since the behavior of TS changed to wrap fields with last_over_time before being handled with non-time-series aggregations, its results would also be different, so I modified the query to produce the same results as before

Probably worth adding back in the other query as a second test, with different output. We can do that in a follow up though. (I think we need to really go over the CSV tests during feature freeze and make sure our coverage is good)

kkrik-es · 2025-09-30T11:05:54Z

...in/esql-core/src/main/java/org/elasticsearch/xpack/esql/core/expression/TypeResolutions.java

     * @see DataType#isRepresentable(DataType)
     */
    public static TypeResolution isRepresentableExceptCounters(Expression e, String operationName, ParamOrdinal paramOrd) {
        return isType(e, DataType::isRepresentable, operationName, paramOrd, "any type except counter types");


Do we still need this and isRepresentableExceptCountersAndSpatial?

It is technically being used in MvSort (which I didn't need to modify for these tests interestingly), but it would definitely make sense to use to use isRepresentableExceptCountersAndAggregateMetricDouble there
The latter is unused indeed though
I can remove in a follow-up

nevermind i removed because i had to fix merge conflicts

kkrik-es

Looks good! Will leave it to Mark or Nhat to stamp.

not-napoleon

LGTM.

not-napoleon · 2025-09-30T13:24:41Z

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java

+        /**
+         * Enable aggregate_metric_double in non-snapshot builds
+         */
+        AGGREGATE_METRIC_DOUBLE_V0,


I am not familiar with this pattern of deprecating capabilities and replacing them with a new one. Where did this come from?

This was the same format as what we did for the TS command so I just mimicked what was there

not-napoleon · 2025-09-30T14:12:17Z

.../org/elasticsearch/xpack/esql/expression/predicate/operator/comparison/EqualsErrorTests.java

-        "boolean, cartesian_point, cartesian_shape, datetime, date_nanos, double, geo_point, geo_shape, integer, ip, keyword, long,"
-            + " semantic_text, text, unsigned_long or version";
+        "boolean, cartesian_point, cartesian_shape, date_nanos, datetime, double, geo_point, geo_shape, geohash, geohex, geotile, "
+            + "integer, ip, keyword, long, text, unsigned_long or version";


Is this supposed to be included? It looks unrelated. Likewise, in NotEqualsErrorTests

Yes this was intentional (same with NotEqualsErrorTest); this code path wasn't being hit prior, but taking AggregateMetricDouble out of under_construction pulled it into these tests and exposed that the error message was ever so slightly different for whatever reason

not-napoleon · 2025-09-30T14:16:05Z

x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test/esql/46_downsample.yml

      esql.query:
        body:
-          query: "TS test-* | STATS max = max(k8s.pod.network.rx)  | LIMIT 100"
+          query: "FROM test-* | STATS max = max(k8s.pod.network.rx)  | LIMIT 100"


Probably worth adding back in the other query as a second test, with different output. We can do that in a follow up though. (I think we need to really go over the CSV tests during feature freeze and make sure our coverage is good)

dnhatn

LGTM, thanks Larisa!

limotova · 2025-09-30T23:09:34Z

All tests passed, just status wasn't reported for one of them, so I am merging

limotova added test-full-bwc Trigger full BWC version matrix tests test-release Trigger CI checks against release build labels Sep 22, 2025

elasticsearchmachine added the v9.2.0 label Sep 22, 2025

limotova force-pushed the remove-agg-metric-feature-flag branch 2 times, most recently from 1caa2f6 to e5f891d Compare September 22, 2025 17:50

limotova force-pushed the remove-agg-metric-feature-flag branch from e5f891d to 2d7876c Compare September 30, 2025 04:43

limotova force-pushed the remove-agg-metric-feature-flag branch 2 times, most recently from a26322e to 77110f6 Compare September 30, 2025 05:58

[ES|QL] Enable aggregate_metric_double outside of snapshots

54d4da8

limotova force-pushed the remove-agg-metric-feature-flag branch from 282100e to 54d4da8 Compare September 30, 2025 09:36

limotova added >non-issue :StorageEngine/ES|QL Timeseries / metrics / logsdb capabilities in ES|QL labels Sep 30, 2025

limotova marked this pull request as ready for review September 30, 2025 09:42

elasticsearchmachine added the Team:StorageEngine label Sep 30, 2025

limotova requested review from dnhatn, kkrik-es and not-napoleon September 30, 2025 10:40

limotova commented Sep 30, 2025

View reviewed changes

kkrik-es reviewed Sep 30, 2025

View reviewed changes

alex-spies mentioned this pull request Sep 30, 2025

ESQL: Update mechanism for under construction data types #135697

Merged

not-napoleon approved these changes Sep 30, 2025

View reviewed changes

dnhatn approved these changes Sep 30, 2025

View reviewed changes

limotova added 4 commits September 30, 2025 09:24

Merge branch 'main' into remove-agg-metric-feature-flag

c6d8f48

spotless apply and other changes

988d892

attempt plugging in transport version

aa942c9

run generateTransportVersion

75e42a8

fix inline stats csv test

0f292be

limotova merged commit 0f2b336 into elastic:main Sep 30, 2025
30 of 31 checks passed

limotova deleted the remove-agg-metric-feature-flag branch September 30, 2025 23:10

[ES|QL] Enable aggregate_metric_double outside of snapshots #135213

[ES|QL] Enable aggregate_metric_double outside of snapshots #135213

Uh oh!

Conversation

limotova commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 30, 2025

ℹ️ Important: Docs version tagging

When to use applies_to tags:

What NOT to do:

🤔 Need help?

Uh oh!

elasticsearchmachine commented Sep 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kkrik-es left a comment

Choose a reason for hiding this comment

Uh oh!

not-napoleon left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dnhatn left a comment

Choose a reason for hiding this comment

Uh oh!

limotova commented Sep 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

limotova commented Sep 22, 2025 •

edited

Loading