Skip to content

Conversation

mosche
Copy link
Contributor

@mosche mosche commented Sep 17, 2025

Upgrade the bundled JDK to 25.
No need to update verification data anymore for openjdk.

Relates to ES-12404

@mosche mosche requested a review from a team September 17, 2025 14:45
@mosche mosche requested a review from a team as a code owner September 17, 2025 14:45
@mosche mosche added >upgrade auto-backport Automatically create backport pull requests when merged v8.19.5 v9.1.5 v8.18.8 v9.0.8 labels Sep 17, 2025
@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label v9.2.0 labels Sep 17, 2025
@mosche mosche added the :Delivery/Build Build or test infrastructure label Sep 17, 2025
@elasticsearchmachine elasticsearchmachine added Team:Delivery Meta label for Delivery team and removed needs:triage Requires assignment of a team area label labels Sep 17, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-delivery (Team:Delivery)

@mosche
Copy link
Contributor Author

mosche commented Sep 17, 2025

Pending on #134719 to silence some noisy timezone id warnings at startup due to a log4j2 bug.

@rjernst
Copy link
Member

rjernst commented Sep 17, 2025

@breskeby does updating the bundled jdk still require updating verification metadata?

@mosche
Copy link
Contributor Author

mosche commented Sep 17, 2025

It looks like it's not necessary anymore due to below:

<trusted-artifacts>
<trust group="beats" name="filebeat"/>
<trust group="beats" name="filebeat-fips"/>
<trust group="beats" name="metricbeat"/>
<trust group="beats" name="metricbeat-fips"/>
<trust group="elasticsearch-distribution" name="elasticsearch"/>
<trust group="org.elasticsearch"/>
<trust group="org.elasticsearch.distribution.zip" name="elasticsearch"/>
<trust group="org.elasticsearch.ml"/>
<trust group="org.elasticsearch.plugin"/>
<trust file=".*-javadoc[.]jar" regex="true"/>
<trust file=".*-sources[.]jar" regex="true"/>
<trust group="openjdk_.*" regex="true"/>
</trusted-artifacts>

@breskeby
Copy link
Contributor

@rjernst as @mosche is right, its not required anyore.

Copy link
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mark-vieira
Copy link
Contributor

Might have to have someone from @elastic/es-analytical-engine take a look at the ES|QL failures. The root cause seems to be for some reason we're throwing an ArithmeticException with a null message when hitting an overflow. To reproduce this locally I have to run the entire test suite. And interestingly enough I've found that muting testEvaluateInManyThreads causes things to pass. Not sure what that means.

@mark-vieira mark-vieira self-assigned this Sep 17, 2025
@mosche mosche added test-windows Trigger CI checks on Windows test-fips Trigger CI checks for FIPS labels Sep 18, 2025
@ChrisHegarty
Copy link
Contributor

Are we talking about ArithmeticException::getMessage returning null - which is allowable within its specification, right?

It may be that the behaviour has changed in JDK 25, but this method could always have returned a null, right? In which case, the caller (ESQL here) should handle that scenario. However, that said, it does seem like a bad behavioural change in JDK 25 if this is now the case.

task.systemProperty('java.security.policy', String.format(Locale.ROOT, "=%s", fipsPolicy))
task.systemProperty('javax.net.ssl.trustStore', fipsTrustStore)
task.systemProperty('org.bouncycastle.fips.approved_only', 'true')
task.systemProperty('jdk.includeInExceptions', 'hostInfo')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rjernst this fixes a category of failures on FIPS, mostly around URI parsing, where the returned exception messages lack specifics that might leak sensitive information (i.e. "illegal character at index n"). This behavior seems to be enabled in "FIPS mode", but can be overriden via a system property (see code change). Is this actually going to be problematic? Do we need to set this system property by default in ES so that we get the same behavior even when running on FIPS, since we do our own redaction of URIs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an example failure: https://gradle-enterprise.elastic.co/s/z75k5utsjqnxu/tests/task/:x-pack:plugin:sql:sql-client:test/details/org.elasticsearch.xpack.sql.client.UriUtilsTests/testUriRedactionInvalidFragment?top-execution=1

Essentially, in the FIPS mode we cannot parse where the problematic characters are from the thrown Exception (the JDK omits this for security reasons) so we return a generic <REDACTED> instead. It's worth noting that we only hit this a) in FIPS mode (or at least as we configure it for testing) and b) when we are parsing an invalid URI. So presumably this is so we don't leak credentials with logs and error messages. I think the end result is when running on FIPS if folks provide bad URIs in configuration or other requests, they might get a slightly less useful error message unless we pass the system property above. Thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps @tvernum can weigh in here. I think it would be ok to be consistent at the expense of less useful error messages, but I could go either way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't quite follow how "fips mode" changes the behaviour (that is, what about our fips tests triggers the change)

From the JDK 25 docs it sounds like the JDK never includes this information without the system property, but your description seem to imply otherwise.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I think I worked it out.

The set of possibly values for the jdk.includeInExceptions property has expanded between JDK24 and JDK25

  • In JDK 24, the options were hostInfo and jar and the default was empty.
  • In JDK 25, the options are hostInfo, hostInfoExclSocket, jar and userInfo and the "default" is hostInfoExclSocket but the default is actually configured in conf/security/java.properties

Our FIPS tests run with a custom security config (that's how you enable FIPS, so we need to copy that new default into our FIPS setup

diff --git i/build-tools-internal/src/main/resources/fips_java.security w/build-tools-internal/src/main/resources/fips_java.security
index 1018b1b96ac..6bed997bfe8 100644
--- i/build-tools-internal/src/main/resources/fips_java.security
+++ w/build-tools-internal/src/main/resources/fips_java.security
@@ -49,3 +49,7 @@ jdk.xml.dsig.secureValidationPolicy=\
     noRetrievalMethodLoops
 jceks.key.serialFilter = java.base/java.lang.Enum;java.base/java.security.KeyRep;\
   java.base/java.security.KeyRep$Type;java.base/javax.crypto.spec.SecretKeySpec;!*
+
+# Needed since JDK25 to match the default config in the out-of-the-box java.security
+jdk.includeInExceptions=hostInfoExclSocket
+
diff --git i/build-tools-internal/src/main/resources/fips_java_oracle.security w/build-tools-internal/src/main/resources/fips_java_oracle.security
index 875151a058e..c50f85202a1 100644
--- i/build-tools-internal/src/main/resources/fips_java_oracle.security
+++ w/build-tools-internal/src/main/resources/fips_java_oracle.security
@@ -50,3 +50,7 @@ jdk.xml.dsig.secureValidationPolicy=\
     noRetrievalMethodLoops
 jceks.key.serialFilter = java.base/java.lang.Enum;java.base/java.security.KeyRep;\
   java.base/java.security.KeyRep$Type;java.base/javax.crypto.spec.SecretKeySpec;!*
+
+# Needed since JDK25 to match the default config in the out-of-the-box java.security
+jdk.includeInExceptions=hostInfoExclSocket
+

(From my testing that doesn't break older JDKs, and it shouldn't, they just ignore unrecognized options)

It's possible we need to also add it to the versions in test/test-clusters/src/main/resources/fips/ as well.

But if we do that, then we don't need this system-property change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reference this is the effective set of differences for java.security between Oracle's OpenJDK v24 and v25

$ diff <(grep '^[^#]' ~/.gradle/jdks/oracle_corporation-24-aarch64-os_x.2/jdk-24.jdk/Contents/Home/conf/security/java.security) <(grep '^[^#]' ~/.gradle/jdks/oracle_corporation-25-aarch64-os_x.2/jdk-25.jdk/Contents/Home/conf/security/java.security)
40c40,41
<     ECDH, TLS_RSA_*
---
>     ECDH, TLS_RSA_*, rsa_pkcs1_sha1 usage HandshakeSignature, \
>     ecdsa_sha1 usage HandshakeSignature, dsa_sha1 usage HandshakeSignature
64a66
> jdk.includeInExceptions=hostInfoExclSocket
66c68
< jdk.security.caDistrustPolicies=SYMANTEC_TLS,ENTRUST_TLS
---
> jdk.security.caDistrustPolicies=SYMANTEC_TLS,ENTRUST_TLS,CAMERFIRMA_TLS
68a71
> jdk.epkcs8.defaultAlgorithm=PBEWithHmacSHA256AndAES_128

I don't think any other other changes are going to be a problem between FIPS/non-FIPS.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tvernum for having a look 👍 I'll push the suggested changes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tvernum Actually, wondering, are you ok doing this as a follow up? I don't want to risk further delays on this PR and would rather isolate the change in a follow up if you're ok.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, as discussed I've included above change, see 63fce46

@mosche mosche merged commit 4aa0f16 into elastic:main Sep 19, 2025
46 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.19 Commit could not be cherrypicked due to conflicts
9.1 Commit could not be cherrypicked due to conflicts
8.18 Commit could not be cherrypicked due to conflicts
9.0 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 134910

mosche added a commit to mosche/elasticsearch that referenced this pull request Sep 19, 2025
Co-authored-by: Mark Vieira <[email protected]>
(cherry picked from commit 4aa0f16)

# Conflicts:
#	x-pack/plugin/esql/build.gradle
mosche added a commit to mosche/elasticsearch that referenced this pull request Sep 19, 2025
Co-authored-by: Mark Vieira <[email protected]>
(cherry picked from commit 4aa0f16)

# Conflicts:
#	x-pack/plugin/esql/build.gradle
#	x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/custom/CustomRequestManagerTests.java
#	x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/custom/request/CustomRequestTests.java
mosche added a commit to mosche/elasticsearch that referenced this pull request Sep 19, 2025
Co-authored-by: Mark Vieira <[email protected]>
(cherry picked from commit 4aa0f16)

# Conflicts:
#	x-pack/plugin/esql/build.gradle
@mosche
Copy link
Contributor Author

mosche commented Sep 19, 2025

💚 All backports created successfully

Status Branch Result
9.1
9.0
8.19
8.18

Questions ?

Please refer to the Backport tool documentation

mosche added a commit to mosche/elasticsearch that referenced this pull request Sep 19, 2025
Co-authored-by: Mark Vieira <[email protected]>
(cherry picked from commit 4aa0f16)

# Conflicts:
#	x-pack/plugin/esql/build.gradle
#	x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/custom/CustomRequestManagerTests.java
#	x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/custom/request/CustomRequestTests.java
mosche added a commit that referenced this pull request Sep 22, 2025
Co-authored-by: Mark Vieira <[email protected]>
(cherry picked from commit 4aa0f16)
mosche added a commit that referenced this pull request Sep 22, 2025
Co-authored-by: Mark Vieira <[email protected]>
(cherry picked from commit 4aa0f16)

# Conflicts:
#	x-pack/plugin/esql/build.gradle
mosche added a commit that referenced this pull request Sep 22, 2025
Co-authored-by: Mark Vieira <[email protected]>
(cherry picked from commit 4aa0f16)

# Conflicts:
#	x-pack/plugin/esql/build.gradle
mosche added a commit that referenced this pull request Sep 22, 2025
* Update bundled JDK to 25 (#134910)

Co-authored-by: Mark Vieira <[email protected]>
(cherry picked from commit 4aa0f16)

# Conflicts:
#	x-pack/plugin/esql/build.gradle
#	x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/custom/CustomRequestManagerTests.java
#	x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/custom/request/CustomRequestTests.java

* fix asm version
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged backport pending :Delivery/Build Build or test infrastructure Team:Delivery Meta label for Delivery team test-fips Trigger CI checks for FIPS test-windows Trigger CI checks on Windows >upgrade v8.18.8 v8.19.5 v9.0.8 v9.1.5 v9.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants