Skip to content

Conversation

@gmarouli
Copy link
Contributor

@gmarouli gmarouli commented Oct 29, 2025

In elastic/elasticsearch#136813 we introduce one more configuration option in the downsample API that allows a user to choose the sampling method between:

  • aggregate: when supported, this will downsample a metric by summarising its values in a aggregate document. This remains the default
  • last_value: this will keep the last value and discard the rest.
POST /my-time-series-index/_downsample/my-downsampled-time-series-index
{
  "fixed_interval": "1d",
  "sampling_method": "last_value"
}

The issue here is that the sampling method will be introduced in the data stream lifecycle in a different way, see elastic/elasticsearch#137023:

PUT _data_stream/my-ds/_lifecycle
{
  "data_retention": "10d",
  "downsampling_method": "last_value",
  "downsampling": [
     {
       "after": "1d",
       "fixed_interval": "5m
     }
  ]
}

In the elasticsearch specification we used the same type to capture the fixed_interval in both, but now we see that these have diverged, so we need to split them into two types in order to preserve correctness.

This is an alternative to #5496.

@gmarouli gmarouli requested a review from pquentin October 29, 2025 08:15
@gmarouli gmarouli added breaking skip-backport This pull request should not be backported labels Oct 29, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 29, 2025

Following you can find the validation changes against the target branch for the API.

API Status Request Response
indices.downsample 🔴 → 🟢 40/42 → 42/42 42/42

You can validate this API yourself by using the make validate target.

@gmarouli gmarouli changed the title Downsampling/add new sampling method v2 Add new sampling method to the downsample API Oct 30, 2025
Copy link
Member

@pquentin pquentin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the careful discussions that led to this! LGTM.

@gmarouli gmarouli merged commit 4f80f1f into main Oct 30, 2025
8 checks passed
@gmarouli gmarouli deleted the downsampling/add-new-sampling-method-v2 branch October 30, 2025 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking skip-backport This pull request should not be backported specification

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants