[CI] CohereServiceUpgradeIT testRerank {upgradedNodes=1} failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-intake #16934 / 8.19.0_bwc-snapshots
• elasticsearch-intake #16932 / 8.19.0_bwc-snapshots

Reproduction Line:

./gradlew ":x-pack:plugin:inference:qa:rolling-upgrade:v8.19.0#bwcTest" -Dtests.class="org.elasticsearch.xpack.application.CohereServiceUpgradeIT" -Dtests.method="testCohereEmbeddings {upgradedNodes=1}" -Dtests.seed=29DDAAF8DC542DE8 -Dtests.bwc=true -Dtests.locale=szl-PL -Dtests.timezone=America/Puerto_Rico -Druntime.java=23

Applicable branches:
9.0

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.RuntimeException: An error occurred while checking cluster 'test-cluster' status.

Issue Reasons:

• [9.0] 2 consecutive failures in step 8.19.0_bwc-snapshots
• [9.0] 2 consecutive failures in pipeline elasticsearch-intake
• [9.0] 2 failures in test testRerank {upgradedNodes=1} (4.2% fail rate in 48 executions)
• [9.0] 2 failures in step 8.19.0_bwc-snapshots (13.3% fail rate in 15 executions)
• [9.0] 2 failures in pipeline elasticsearch-intake (18.2% fail rate in 11 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch 9.0

Mute Reasons:

• [9.0] 2 consecutive failures in step 8.19.0_bwc-snapshots
• [9.0] 2 consecutive failures in pipeline elasticsearch-intake
• [9.0] 2 failures in test testRerank {upgradedNodes=1} (4.2% fail rate in 48 executions)
• [9.0] 2 failures in step 8.19.0_bwc-snapshots (13.3% fail rate in 15 executions)
• [9.0] 2 failures in pipeline elasticsearch-intake (18.2% fail rate in 11 executions)

Build Scans:

• elasticsearch-intake #16934 / 8.19.0_bwc-snapshots
• elasticsearch-intake #16932 / 8.19.0_bwc-snapshots

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)

[tlrx]

I suspect this test failed because of the reason I explain in this #121365 (comment)