Bump elasticsearch-java to version 9.x

[ramanathan1504]

#3663 This pull request updates the dependencies in the log4j-layout-template-json-test/pom.xml file to keep the project up to date with the latest Elastic Stack versions and adds a new test dependency. The most important changes are:

Dependency updates:

• Bumped the elastic.version property from 8.17.3 to 9.1.3 to align with the latest Elastic Stack release.

New test dependency:

• Added the elasticsearch-rest-client library (version 8.13.4) as a test-scoped dependency to support testing with the Elasticsearch REST client.

[github-actions]

Job	Requested goals	Build Tool Version	Build Outcome	Build Scan®
build-macos-latest	clean install	3.9.8	✅
build-ubuntu-latest	clean install	3.9.8	✅
build-windows-latest	clean install	3.9.8	✅

Generated by gradle/develocity-actions

[ppkarwasz]

Hi @ramanathan1504, thanks for the PR!

After checking the Elasticsearch Java client 9.0.0 release notes:

• The Java baseline is now JRE 17. Our CI runs tests on JRE 8, so the integration test will fail there even if it passes locally (you can reproduce locally by setting CI=true).
• The legacy RestClient was deprecated in favor of Rest5Client (based on Apache HttpComponents 5.x, since version 4.x is EOL).

Proposal

1. Upgrade main to use Rest5Client.

2. Keep the 2.x branch on Elasticsearch 8.x and update .github/dependabot.yaml to ignore 9.x (and above) proposals.

3. Add a reminder to drop 8.x when it goes EOL in ~2 years (see the EOL policy) by annotating the test class with JUnit Pioneer:

   @FailAt(
       date = "2027-07-15",
       reason = "Elasticsearch 8.x is no longer supported"
   )

[ramanathan1504]

@ppkarwasz
Thank you for the feedback
I will close this PR and create new ones and follow your proposal.

[vy]

@ramanathan1504, please don't close PRs and start a new one each time – it makes it impossible to track the evolution of the eventual PR.

[ppkarwasz]

I agree: the changes to .github/dependabot.yaml must be done on the default branch, which is 2.x, so they can be done in this PR.

The @FailAt annotation also needs to be done in 2.x.

Only the upgrade to Rest5Client needs a separate PR.

[vy]

I think

1. we're comparing apples with oranges. We use ES client to test a functionality, the fact that it depends on Java 17 should not stop us from using it, in particular, given that we already use Java 17 to compile the sources. That is, we already have Java 17 at our disposal while running tests – saying "Don't use Java 17 because our CI tests run in Java 8" does not make sense to me.
2. There is only one problematic test: LogstashIT. It is executed via maven-failsafe-plugin configured in the docker profile defined in log4j-layout-template-json-test/pom.xml, it has nothing to do with java8-tests profile in log4j-parent/pom.xml, and the latter forces Java 8.
3. Using different major versions across 2.x and main will obstruct porting process.

@ppkarwasz, what about the following plan instead?

1. Use the latest and greatest ES client on 2.x
2. Update maven-failsafe-plugin in the docker profile in log4j-layout-template-json-test/pom.xml to force Java 17

[ppkarwasz]

Sounds good!

BTW, I was also considering dropping tests on Java 8, since the difference between JRE 8 and 9 is very painful.

[ramanathan1504]

@ppkarwasz @vy
I am reopened the pr, also little confused if anyone of you describe what's needed to do?

[vy]

@ramanathan1504, you can proceed with the following plan:

1. Use the latest and greatest ES client on 2.x
2. Update maven-failsafe-plugin in the docker profile in log4j-layout-template-json-test/pom.xml to force Java 17

[ramanathan1504]

@vy
ok

[ppkarwasz]

Hi @ramanathan1504,

Looks good to me, except the two remarks below.

[ppkarwasz]

@ramanathan1504,

You lost some signatures, can you rebase with git rebase -S?

[ramanathan1504]

@ppkarwasz
ok i will do that

[ramanathan1504]

@ppkarwasz
commits all are verified

[ppkarwasz]

LGTM, thanks! 💯

[ramanathan1504]

@vy
Is this right?

recent 8.x version of the modern client library (co.elastic.clients:elasticsearch-java) for the Java code.

the new 9.x version for the Docker images of Elasticsearch and Logstash.

[vy]

@vy Is this right?

recent 8.x version of the modern client library (co.elastic.clients:elasticsearch-java) for the Java code.

the new 9.x version for the Docker images of Elasticsearch and Logstash.

No, use ES 9 everywhere. We use ES only for integration tests and, as @ppkarwasz stated earlier (I cannot find the commit associated with his message, since you force-pushed changes, but the conversation is still visible), ITs run using Java 17 anyway.

[ramanathan1504]

@vy Is this right?
recent 8.x version of the modern client library (co.elastic.clients:elasticsearch-java) for the Java code.
the new 9.x version for the Docker images of Elasticsearch and Logstash.

No, use ES 9 everywhere. We use ES only for integration tests and, as @ppkarwasz stated earlier (I cannot find the commit associated with his message, since you force-pushed changes, but the conversation is still visible), ITs run using Java 17 anyway.
@vy
am reverted back the LogstashIT to old and bumped the version only

[vy]

@ramanathan1504, after several review rounds, I could not succeed in making you remove redundant dependencies and unrelated changes. I eventually carried them out myself in 561107e. Please be more attentive to these details next time – it will make life easier for both of us.

[ramanathan1504]

Really sorry for this,i will take care these things in future and i will retrospect this pr and comments..next time will do the good one thanks @vy

[vy]

@ramanathan1504, would you submit a PR porting this to main, please? (Bump the version to 9.1.5 instead.)

[ramanathan1504]

@vy sure i will do
thanks

[ramanathan1504]

@vy can you review the pr #3951