[Test] Replace 0.0.0 version with "detached" property on ElasticsearchDistibution #134584

jozala · 2025-09-11T21:36:17Z

Introduce a new property to indicate if the version is a custom build not tied to any release. This was previously handled by specifying version as 0.0.0, but caused problems when comparing versions (e.g., to decide on configuration).

In the tests, which support the bcUpgradeTest task it means that they need to:

apply the elasticsearch.bc-upgrade-test (this was already required),
configure the version of the cluster/node using the ElasticsearchCluster JUnit rule (It was already required, but after this change it can be read from tests.bwc.main.version or tests.old_cluster_version since both are the same. Previously, sometimes version had to be applied from tests.old_cluster_version which could be 0.0.0 to be able to support special cases.)
set detachedVersion to true using the ElasticsearchCluster JUnit rule when the cluster should resolve to the version not tied to any released version nor the current HEAD (This is new. This setting is currently tied to the existence of the tests.bwc.refspec.main system property).

I haven't found any of the legacy rest/yaml tests supporting bcUpgradeTest (none applies elasticsearch.bc-upgrade-test plugin), so I haven't been implementing any support for detached version for the legacy test framework.

jozala · 2025-09-19T14:44:18Z

build-tools-internal/src/main/groovy/elasticsearch.bc-upgrade-test.gradle

+  doFirst {
+    if (System.getProperty("tests.bwc.refspec.main") == null) {
+      throw new GradleException("You must set the `tests.bwc.refspec.main` system property to run bcUpgradeTest")
+    }
+    if (System.getProperty("tests.bwc.main.version") == null) {
+      throw new GradleException("You must set the `tests.bwc.main.version` system property to run bcUpgradeTest")
+    }
+  }
+  if (System.getProperty("tests.bwc.refspec.main") != null && System.getProperty("tests.bwc.main.version") != null) {
+    usesBwcDistributionFromRef(System.getProperty("tests.bwc.refspec.main"), Version.fromString(System.getProperty("tests.bwc.main.version")))
+    systemProperty("tests.old_cluster_version", System.getProperty("tests.bwc.main.version"))
+  }


This and usesBwcDistributionFromRef implementation in RestTestBasePlugin are the core changes of this PR. Instead of setting the 0.0.0 version, the correct version from required tests.bwc.main.version is used.

jozala · 2025-09-19T14:48:21Z

...s-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/RestTestBasePlugin.java

+                    if (version.before(bwcVersions.getMinimumWireCompatibleVersion())) {
                        // If we are upgrade testing older versions we also need to upgrade to 7.last


The above usesBwcDistribution is now simplified, because handling of 0.0.0 (now "detached") has been moved to the usesBwcDistributionFromRef implementation below.

jozala · 2025-09-19T14:50:58Z

build-tools/src/main/java/org/elasticsearch/gradle/ElasticsearchDistribution.java

+    /**
+     * Informs if the version is not tied to any Elasticsearch release and is a custom build.
+     * This is true when the distribution is not from HEAD but also not any known released version.
+     * In that case the detached source build needs to be prepared by `usedBwcDistributionFromRef(ref, version)`.
+     */
+    public boolean isDetachedVersion() {
+        return detachedVersion.get();
+    }
+
+    public void setDetachedVersion(boolean detachedVersion) {
+        this.detachedVersion.set(detachedVersion);
+    }
+


I refrained from using "snapshot" and used "detached" instead, because "snapshot" seems to be overloaded term. However, I'm interested in your opinion on that and any suggestions for better naming.

I like the term detatched. I certainly can't think of a better one and I agree "snapshot" is both overloaded and not really appropriate here.

jozala · 2025-09-19T14:52:29Z

qa/full-cluster-restart/build.gradle

+  doFirst {
+    if (System.getProperty("tests.bwc.refspec.main") == null) {
+      throw new GradleException("You must set the `tests.bwc.refspec.main` system property to run luceneBwcTest")
+    }
+    if (System.getProperty("tests.bwc.main.version") == null) {
+      throw new GradleException("You must set the `tests.bwc.main.version` system property to run luceneBwcTest")
+    }
+  }
+  if (System.getProperty("tests.bwc.refspec.main") != null && System.getProperty("tests.bwc.main.version") != null) {
+    usesBwcDistributionFromRef(System.getProperty("tests.bwc.refspec.main"), Version.fromString(System.getProperty("tests.bwc.main.version")))
+    systemProperty("tests.old_cluster_version", System.getProperty("tests.bwc.main.version"))
+  }


Same logic change as in /elasticsearch.bc-upgrade-test.gradle

jozala · 2025-09-19T14:54:53Z

...t-clusters/src/main/java/org/elasticsearch/test/cluster/local/DefaultLocalClusterHandle.java

        Node node = nodes.get(index);
        node.stop(false);
        LOGGER.info("Upgrading node '{}' to version {}", node.getName(), version);
+        node.getSpec().setDetachedVersion(false);


This is needed for the cluster to be upgraded to the actual specified version and not keep using the same detached version as before.
It has a limitation of not being able to upgrade to the detached version, but there is no such test case currently.

Yeah, this separation is what makes this somewhat awkward. Which is why I originally envisioned the notion of "detached" here to be captured by Version itself. All the test clusters stuff uses it's own Version class, so we could add a detached property to it. It would save us having to muck with all these method signatures and API that already accept a Version while still being able to pass that knowledge down to things that care about that.

So how about we add detached to Version. We'll then add some factory methods that take the additional boolean argument. We can do the same on LocalSpecBuilder by adding a version(String, boolean) method. If unspecified the default will be to set detached = false. That way this all "just works". We just look at the detached property of the passed Version to figure out what to do. This also allows us in the future to have tests that upgrade to detached versions as well. As this is written that wouldn't be possible.

elasticsearchmachine · 2025-09-19T14:56:37Z

Pinging @elastic/es-delivery (Team:Delivery)

ElasticsearchDistibution for tests Introduce a new property to indicate if the version is a custom build not tied to any release. This was previously handled by specifying version as 0.0.0, but caused problems when comparing versions (e.g., to decide on configuration).

Added `.detachedVersion(System.getProperty("tests.bwc.refspec.main") != null)` to all the tests which apply 'elasticsearch.bc-upgrade-test' plugin to make them properly resolve distribution (previously configured by specifying `0.0.0` version).

In the bcUpgradeTests these tests were previously skipped because version was set to `0.0.0` which was outside the supported range. Now, when `tests.old_cluster_version` has correct version they run, but fail just because `9.2.0-SNAPSHOT` was considered unsupported version format with the "SNAPSHOT".

Needs to stay there, because of the serverless tests.

…hDistibution (elastic#134584) Introduce a new `detached` property to indicate if the version is a custom build not tied to any release. This was previously handled by specifying version as 0.0.0, but caused problems when comparing versions (e.g., to decide on configuration).

elasticsearchmachine · 2025-10-03T11:15:18Z

💔 Backport failed

Status	Branch	Result
❌	8.19	Commit could not be cherrypicked due to conflicts
✅	9.1
❌	9.0	Commit could not be cherrypicked due to conflicts
❌	8.18	Commit could not be cherrypicked due to conflicts
✅	9.2

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 134584

…hDistibution (elastic#134584) Introduce a new `detached` property to indicate if the version is a custom build not tied to any release. This was previously handled by specifying version as 0.0.0, but caused problems when comparing versions (e.g., to decide on configuration).

…hDistibution (#134584) (#135903) Introduce a new `detached` property to indicate if the version is a custom build not tied to any release. This was previously handled by specifying version as 0.0.0, but caused problems when comparing versions (e.g., to decide on configuration).

…hDistibution (#134584) (#135904) Introduce a new `detached` property to indicate if the version is a custom build not tied to any release. This was previously handled by specifying version as 0.0.0, but caused problems when comparing versions (e.g., to decide on configuration).

…hDistibution (elastic#134584) Introduce a new `detached` property to indicate if the version is a custom build not tied to any release. This was previously handled by specifying version as 0.0.0, but caused problems when comparing versions (e.g., to decide on configuration). (cherry picked from commit 664a604) # Conflicts: # qa/rolling-upgrade/src/javaRestTest/java/org/elasticsearch/upgrades/AbstractRollingUpgradeWithSecurityTestCase.java # test/framework/src/main/java/org/elasticsearch/test/rest/ESRestTestCase.java # x-pack/plugin/logsdb/qa/rolling-upgrade/src/javaRestTest/java/org/elasticsearch/upgrades/StandardToLogsDbIndexModeRollingUpgradeIT.java

jozala · 2025-10-03T13:22:47Z

💔 Some backports could not be created

Status	Branch	Result
✅	9.0
❌	8.19	Conflict resolution was aborted by the user
❌	8.18	Conflict resolution was aborted by the user

Manual backport

To create the backport manually run:

backport --pr 134584

Questions ?

Please refer to the Backport tool documentation

jozala · 2025-10-03T13:52:02Z

This doesn't need to be backported to 8.19 and 8.18 because bcUpgradeTest and version 0.0.0 was never introduced on these branches.

…hDistibution (#134584) (#135924) Introduce a new `detached` property to indicate if the version is a custom build not tied to any release. This was previously handled by specifying version as 0.0.0, but caused problems when comparing versions (e.g., to decide on configuration). (cherry picked from commit 664a604) # Conflicts: # qa/rolling-upgrade/src/javaRestTest/java/org/elasticsearch/upgrades/AbstractRollingUpgradeWithSecurityTestCase.java # test/framework/src/main/java/org/elasticsearch/test/rest/ESRestTestCase.java # x-pack/plugin/logsdb/qa/rolling-upgrade/src/javaRestTest/java/org/elasticsearch/upgrades/StandardToLogsDbIndexModeRollingUpgradeIT.java

These asserts have been enabled for the lucene compat because `oldVersion` is now correctly set after elastic#134584 which removed `0.0.0` version.

jozala added >non-issue >test Issues or PRs that are addressing/adding tests :Delivery/Build Build or test infrastructure auto-backport Automatically create backport pull requests when merged v9.2.0 v8.19.5 v9.1.5 v8.18.8 v9.0.8 labels Sep 11, 2025

jozala force-pushed the detached-property-on-esdistro branch 4 times, most recently from 93d9153 to c0bb3d6 Compare September 16, 2025 12:10

jozala force-pushed the detached-property-on-esdistro branch from b571c5c to 18c721b Compare September 19, 2025 10:05

elasticsearchmachine added the serverless-linked Added by automation, don't add manually label Sep 19, 2025

jozala commented Sep 19, 2025

View reviewed changes

jozala changed the title ~~WIP: [Test] Replace 0.0.0 version with "detached" property on ElasticsearchDistibution~~ [Test] Replace 0.0.0 version with "detached" property on ElasticsearchDistibution Sep 19, 2025

jozala marked this pull request as ready for review September 19, 2025 14:56

jozala requested a review from a team as a code owner September 19, 2025 14:56

elasticsearchmachine added the Team:Delivery Meta label for Delivery team label Sep 19, 2025

jozala requested a review from mark-vieira September 19, 2025 14:56

jozala mentioned this pull request Sep 19, 2025

Limit default allocated processors for the test cluster #133633

Merged

jozala added 3 commits September 19, 2025 17:32

elasticsearchmachine added v8.19.6 v9.0.9 v9.1.6 v9.3.0 and removed v8.19.5 v9.0.8 v9.1.5 v9.2.0 labels Oct 2, 2025

jozala added the v9.2.1 label Oct 2, 2025

jozala added 2 commits October 3, 2025 11:13

Merge branch 'main' into detached-property-on-esdistro

0e831f6

Version 0.0.0 exception reverted

4417727

Needs to stay there, because of the serverless tests.

jozala merged commit 664a604 into elastic:main Oct 3, 2025
34 checks passed

jozala mentioned this pull request Oct 3, 2025

[9.1] [Test] Replace 0.0.0 version with "detached" property on ElasticsearchDistibution (#134584) #135903

Merged

jozala mentioned this pull request Oct 3, 2025

[9.2] [Test] Replace 0.0.0 version with "detached" property on ElasticsearchDistibution (#134584) #135904

Merged

elasticsearchmachine added the backport pending label Oct 3, 2025

jozala mentioned this pull request Oct 3, 2025

[9.0] [Test] Replace 0.0.0 version with "detached" property on ElasticsearchDistibution (#134584) #135924

Merged

jozala removed v8.19.6 v8.18.9 labels Oct 3, 2025

jozala removed the backport pending label Oct 7, 2025

jozala mentioned this pull request Oct 7, 2025

Fix lucene compat tests by keeping asserts disabled #136094

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Test] Replace 0.0.0 version with "detached" property on ElasticsearchDistibution #134584

[Test] Replace 0.0.0 version with "detached" property on ElasticsearchDistibution #134584

jozala commented Sep 11, 2025 •

edited

Loading

Uh oh!

jozala Sep 19, 2025 •

edited

Loading

Uh oh!

jozala Sep 19, 2025

Uh oh!

jozala Sep 19, 2025 •

edited

Loading

Uh oh!

mark-vieira Sep 19, 2025

Uh oh!

jozala Sep 19, 2025

Uh oh!

jozala Sep 19, 2025

Uh oh!

mark-vieira Sep 19, 2025

Uh oh!

elasticsearchmachine commented Sep 19, 2025

Uh oh!

Uh oh!

elasticsearchmachine commented Oct 3, 2025

Uh oh!

jozala commented Oct 3, 2025

Uh oh!

jozala commented Oct 3, 2025

Uh oh!

Uh oh!

		if (version.before(bwcVersions.getMinimumWireCompatibleVersion())) {
		// If we are upgrade testing older versions we also need to upgrade to 7.last

[Test] Replace 0.0.0 version with "detached" property on ElasticsearchDistibution #134584

[Test] Replace 0.0.0 version with "detached" property on ElasticsearchDistibution #134584

Conversation

jozala commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jozala Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jozala Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

jozala Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mark-vieira Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

jozala Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

jozala Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

mark-vieira Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Sep 19, 2025

Uh oh!

Uh oh!

elasticsearchmachine commented Oct 3, 2025

💔 Backport failed

Uh oh!

jozala commented Oct 3, 2025

💔 Some backports could not be created

Manual backport

Questions ?

Uh oh!

jozala commented Oct 3, 2025

Uh oh!

Uh oh!

jozala commented Sep 11, 2025 •

edited

Loading

jozala Sep 19, 2025 •

edited

Loading

jozala Sep 19, 2025 •

edited

Loading