[CI] FullClusterRestartIT class failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #11133 / 8.3.3_bwc
• elasticsearch-intake #29712 / 10.0.0_lucene-compat
• elasticsearch-intake #29706 / 10.0.0_lucene-compat
• elasticsearch-intake #29704 / 10.0.0_lucene-compat
• elasticsearch-intake #29702 / 10.0.0_lucene-compat
• elasticsearch-intake #29682 / 10.0.0_lucene-compat
• elasticsearch-intake #29681 / 10.0.0_lucene-compat
• elasticsearch-intake #29677 / 10.0.0_lucene-compat
• elasticsearch-intake #29657 / 10.0.0_lucene-compat
• elasticsearch-intake #29653 / 10.0.0_lucene-compat

Reproduction Line:

undefined

Applicable branches:
9.2

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

undefined

Issue Reasons:

• [9.2] 15 failures in class org.elasticsearch.upgrades.FullClusterRestartIT (1.5% fail rate in 1000 executions)
• [9.2] 14 failures in step 10.0.0_lucene-compat (29.2% fail rate in 48 executions)
• [9.2] 14 failures in pipeline elasticsearch-intake (29.2% fail rate in 48 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch 9.0

Mute Reasons:

• [9.0] 4 failures in class org.elasticsearch.ingest.geoip.FullClusterRestartIT (1.3% fail rate in 297 executions)
• [9.0] 2 failures in step 8.18.0_bwc-snapshots (1.1% fail rate in 188 executions)
• [9.0] 2 failures in pipeline elasticsearch-pull-request (1.9% fail rate in 107 executions)

Build Scans:

• elasticsearch-intake #17471 / 8.18.0_bwc-snapshots
• elasticsearch-pull-request #54786 / 8.18.0_bwc-snapshots
• elasticsearch-periodic #6106 / openjdk23_8.18.0_java-matrix-bwc
• elasticsearch-pull-request #54240 / 8.19.0_bwc-snapshots

[elasticsearchmachine]

Pinging @elastic/es-delivery (Team:Delivery)

[elasticsearchmachine]

This has been muted on branch 8.x

Mute Reasons:

• [8.x] 3 failures in class org.elasticsearch.xpack.application.FullClusterRestartIT (0.3% fail rate in 944 executions)
• [8.x] 2 failures in pipeline elasticsearch-periodic (20.0% fail rate in 10 executions)

Build Scans:

• elasticsearch-intake #17639 / 8.17.2_bwc-snapshots
• elasticsearch-periodic #6207 / openjdk17_8.17.2_java-matrix-bwc
• elasticsearch-periodic #6200 / openjdk17_8.18.0_java-fips-matrix-bwc

[joegallo]

[9.0] 4 failures in class org.elasticsearch.ingest.geoip.FullClusterRestartIT

The geoip failures have been handled by me on #122537 and I've unmuted them in the backports of that PR. I didn't touch the stuff from the other packages though, they're none of my business.

As an aside I don't think it's necessarily the best behavior that these tests from different packages are getting lumped onto the same ticket (perhaps because they have the same test class name?) -- but in the end it probably happens rarely enough that I could understand if it's a problem that's not worth solving. 🤷

[joegallo]

Then again, looking closer at the specific failures in the linked build scans, I think it's fair to say that I only solved https://gradle-enterprise.elastic.co/s/dzayeoidfhhzo/tests/task/:modules:ingest-geoip:qa:full-cluster-restart:v8.18.0%23bwcTest/details/org.elasticsearch.ingest.geoip.FullClusterRestartIT/testGeoIpSystemFeaturesMigration%20%7Bcluster%3DUPGRADED%7D?top-execution=1 and https://gradle-enterprise.elastic.co/s/naxvfvde6mitk/tests/task/:modules:ingest-geoip:qa:full-cluster-restart:v8.19.0%23bwcTest/details/org.elasticsearch.ingest.geoip.FullClusterRestartIT/testGeoIpSystemFeaturesMigration%20%7Bcluster%3DUPGRADED%7D?top-execution=1 from there, and not the other two cases where the tests failed for other reasons, so maybe it's more fair to include those failing tests on this ticket than I originally assumed. Still 🤷! 😄

[neilbhavsar]

mutedl, needs review

[neilbhavsar]

Re looking at this ticket it seems this it has not been failing since @joegallo 's changes I think we can safely close this one