Skip to content

Commit 231506c

Browse files
committed
Add explanation about DLM using the source sampling method
1 parent 5340685 commit 231506c

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

modules/data-streams/src/main/java/org/elasticsearch/datastreams/lifecycle/DataStreamLifecycleService.java

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -602,6 +602,14 @@ private void downsampleIndexOnce(
602602
IndexMetadata sourceIndexMetadata,
603603
String downsampleIndexName
604604
) {
605+
// When an index is already downsampled with a method, we require all later downsampling rounds to use the same method.
606+
// This is necessary to preserve the relation of the downsampled index to the raw data. For example, if an index is already
607+
// downsampled and downsampled it again to 1 hour; we know that a document represents either the aggregated raw data of an hour
608+
// or the last value of the raw data within this hour. If we mix the methods, we cannot derive any meaning from them.
609+
// Furthermore, data stream lifecycle is configured on the data stream level and not on the individual index level, meaning that
610+
// when a user changes downsampling method, some indices would not be able to be downsampled anymore.
611+
// For this reason, when we encounter an already downsampled index, we use the source downsampling method which might be different
612+
// from the requested one.
605613
var sourceIndexSamplingMethod = DownsampleConfig.SamplingMethod.fromIndexMetadata(sourceIndexMetadata);
606614
String sourceIndex = sourceIndexMetadata.getIndex().getName();
607615
DownsampleAction.Request request = new DownsampleAction.Request(

0 commit comments

Comments
 (0)