[CI] ILMDownsampleDisruptionIT testILMDownsampleRollingRestart failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-intake #30739 / part2
• elasticsearch-pull-request #101786 / part-2-windows
• elasticsearch-pull-request #101848 / part-2
• elasticsearch-pull-request #101842 / part-2
• elasticsearch-pull-request #101839 / part-2
• elasticsearch-pull-request #101838 / part-2
• elasticsearch-pull-request #101836 / part-2
• elasticsearch-intake #30738 / part2
• elasticsearch-pull-request #101824 / part-2
• elasticsearch-pull-request #101822 / part-2

Reproduction Line:

./gradlew ":x-pack:plugin:downsample:internalClusterTest" --tests "org.elasticsearch.xpack.downsample.ILMDownsampleDisruptionIT.testILMDownsampleRollingRestart" -Dtests.seed=63382B2B1F6E4F46 -Dtests.locale=en-SK -Dtests.timezone=Brazil/DeNoronha -Druntime.java=25

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

org.junit.ComparisonFailure: expected:<[last_valu]e> but was:<[aggregat]e>

Issue Reasons:

• [main] 2 consecutive failures in test testILMDownsampleRollingRestart
• [main] 2 consecutive failures in step part2
• [main] 2 consecutive failures in pipeline elasticsearch-intake
• [main] 25 failures in test testILMDownsampleRollingRestart (43.9% fail rate in 57 executions)
• [main] 7 failures in step part2 (70.0% fail rate in 10 executions)
• [main] 2 failures in step part-2-windows (66.7% fail rate in 3 executions)
• [main] 16 failures in step part-2 (37.2% fail rate in 43 executions)
• [main] 7 failures in pipeline elasticsearch-intake (70.0% fail rate in 10 executions)
• [main] 16 failures in pipeline elasticsearch-pull-request (40.0% fail rate in 40 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[nielsbauman]

Had a quick look at failures of the last year: https://es-delivery-stats.elastic.dev/app/r/s/ovqIr and noticed that there's a failure with the same exception on Feb 26, 2025, and there are only 18 occurrences since then. It's also worth noting that the exception occurs during the test cleanup (in ESIntegTestCase.cleanUpCluster). I'll label this as low-risk.

[nielsbauman]

Intuitively, I'm inclined to think this is more related to downsampling itself rather than ILM, so I'm going to forward this to the Storage Engine team for now. Feel free to throw it back to us if it turns out there's anything we can do.

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 5 failures in test testILMDownsampleRollingRestart (2.2% fail rate in 230 executions)
• [main] 4 failures in step part-2 (3.3% fail rate in 120 executions)
• [main] 4 failures in pipeline elasticsearch-pull-request (3.3% fail rate in 120 executions)

Build Scans:

• elasticsearch-periodic-platform-support #11230 / almalinux-8-aarch64_checkpart2_platform-support-arm
• elasticsearch-pull-request #99182 / part-2
• elasticsearch-pull-request #99113 / part-2
• elasticsearch-pull-request #99003 / part-2
• elasticsearch-pull-request #98995 / part-2