[CI] ReindexDatastreamIndexTransportActionIT class failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic-platform-support #6317 / rhel-9_platform-support-unix
• elasticsearch-pull-request #56097 / part-2
• elasticsearch-pull-request #55686 / part-2
• elasticsearch-intake #17562 / part2
• elasticsearch-periodic #6251 / encryption-at-rest
• elasticsearch-pull-request #55650 / part-2
• elasticsearch-pull-request #55646 / part-2
• elasticsearch-intake #17536 / part2
• elasticsearch-pull-request #55589 / part-2
• elasticsearch-periodic-platform-support #6233 / opensuse-leap-15_platform-support-unix

Reproduction Line:

./gradlew ":x-pack:plugin:migrate:internalClusterTest" --tests "org.elasticsearch.xpack.migrate.action.ReindexDatastreamIndexTransportActionIT.testMissingSourceIndex" -Dtests.seed=D491E9037A79861A -Dtests.locale=ff-Latn-LR -Dtests.timezone=Africa/Kigali -Druntime.java=23

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: All incoming requests on node [node_s2] should have finished. Expected 0 bytes for requests in-flight but got 298 bytes; pending tasks [[]]

Issue Reasons:

• [main] 26 failures in class org.elasticsearch.xpack.migrate.action.ReindexDatastreamIndexTransportActionIT (2.9% fail rate in 910 executions)
• [main] 2 failures in step rhel-9_platform-support-unix (11.1% fail rate in 18 executions)
• [main] 3 failures in step part2 (2.3% fail rate in 131 executions)
• [main] 14 failures in step part-2 (4.9% fail rate in 284 executions)
• [main] 2 failures in step release-tests (12.5% fail rate in 16 executions)
• [main] 4 failures in pipeline elasticsearch-periodic-platform-support (20.0% fail rate in 20 executions)
• [main] 4 failures in pipeline elasticsearch-periodic (20.0% fail rate in 20 executions)
• [main] 3 failures in pipeline elasticsearch-intake (2.3% fail rate in 131 executions)
• [main] 14 failures in pipeline elasticsearch-pull-request (4.9% fail rate in 284 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 5 failures in class org.elasticsearch.xpack.migrate.action.ReindexDatastreamIndexTransportActionIT (2.9% fail rate in 175 executions)
• [main] 2 failures in step part-2 (2.9% fail rate in 68 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (2.9% fail rate in 68 executions)

Build Scans:

• elasticsearch-periodic-platform-support #6198 / rhel-9_platform-support-unix
• elasticsearch-periodic #6209 / release-tests
• elasticsearch-pull-request #55093 / part-2
• elasticsearch-pull-request #55083 / part-2
• elasticsearch-intake #17284 / part2

[parkertimmins]

Fixed by #121782

[elasticsearchmachine]

This has been muted on branch 8.x

Mute Reasons:

• [8.x] 6 failures in class org.elasticsearch.xpack.migrate.action.ReindexDatastreamIndexTransportActionIT (3.0% fail rate in 201 executions)
• [8.x] 3 failures in pipeline elasticsearch-periodic-platform-support (60.0% fail rate in 5 executions)

Build Scans:

• elasticsearch-periodic-platform-support #6224 / almalinux-8-aarch64_checkpart2_platform-support-arm
• elasticsearch-pull-request #55364 / part-2
• elasticsearch-periodic-platform-support #6217 / windows-2019_checkpart2_platform-support-windows
• elasticsearch-intake #17297 / part2
• elasticsearch-periodic-platform-support #6196 / oraclelinux-7_platform-support-unix
• elasticsearch-periodic #6207 / openjdk21_checkpart2_java-matrix