Skip to content

Conversation

@ywangd
Copy link
Member

@ywangd ywangd commented Jun 11, 2025

The PR updates ElasticsearchNodeCommandTests so that they test the actual behaviours after multi-project related changes:

  1. Persisted cluster state is always written with multi-project enabled
  2. The CLI tools are expected to run on the same node version, i.e. no BWC
  3. Unknown (unregistered) customs can be read and write by the CLI tools without modification

Resolves: ES-11328

The PR updates ElasticsearchNodeCommandTests so that they test the
right behaviours after multi-project related changes:
1. Persisted cluster state is always written with multi-project enabled
2. The CLI tools are expected to run on the same node version, i.e. no BWC
3. Unknown (unregistered) customs can be read and write by the CLI tools
   without modification

Resolves: ES-11328
@ywangd ywangd added >test Issues or PRs that are addressing/adding tests :Core/Infra/CLI CLI utilities, scripts, and infrastructure v9.1.0 labels Jun 11, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@elasticsearchmachine elasticsearchmachine added the Team:Core/Infra Meta label for core/infra team label Jun 11, 2025
final Metadata latestMetadata = randomMeta();
final XContentBuilder builder = JsonXContent.contentBuilder();
builder.startObject();
Metadata.FORMAT.toXContent(builder, latestMetadata);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no longer correct after MP changes because it serializes metadata without multi-project=true and that is what PersistedClusterStateService does. The CLI tools are never run across versions so that the test needs a serialized state with up-to-date format (by PersistedClusterStateService).

Comment on lines -67 to -73
Metadata loadedMetadata;
try (XContentParser parser = createParser(parserConfig, JsonXContent.jsonXContent, BytesReference.bytes(builder))) {
loadedMetadata = Metadata.fromXContent(parser);
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This round-trip does not seem to directly test ElasticsearchNodeCommand at all. It is replaced directly calling ElasticsearchNodeCommand#loadTermAndClusterState.

Comment on lines -83 to -89
if (preserveUnknownCustoms) {
// check that we reserialize unknown metadata correctly again
final Path tempdir = createTempDir();
Metadata.FORMAT.write(loadedMetadata, tempdir);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The node CLI always perserve unknown customs as long as they are either ClusterCustom, ProjectCustom or Condition. Therefore the PR removes test variations based on the preserveUnknownCustoms boolean. The Metadata.FORMAT.write method has the problem of not using multi-project=true when serializing. It is replaced by using the PersistedClusterStateService created as part of the node CLI.

);
}

private static class TestMissingProjectCustomMetadata extends TestProjectCustomMetadata {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually simulating unknown (unregistered) customs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deserves a comment in the code to that effect - the "missing" is a little confusing cos they're not missing, they're right here, the point is that they're not registered. Except they are registered in some registries, just not the one that the ElasticsearchNodeCommand uses.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah see 691507a

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, much nicer than using IndexGraveyard to stand-in for the unknown custom particularly since I think this wasn't unknown in practice.


static {
Map<String, String> params = Maps.newMapWithExpectedSize(2);
Map<String, String> params = Maps.newMapWithExpectedSize(4);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 tho unrelated to the test changes

);
}

private static class TestMissingProjectCustomMetadata extends TestProjectCustomMetadata {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deserves a comment in the code to that effect - the "missing" is a little confusing cos they're not missing, they're right here, the point is that they're not registered. Except they are registered in some registries, just not the one that the ElasticsearchNodeCommand uses.

@ywangd ywangd added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Jun 13, 2025
@ywangd
Copy link
Member Author

ywangd commented Jun 15, 2025

@elasticmachine update branch

@ywangd
Copy link
Member Author

ywangd commented Jun 16, 2025

@elasticmachine update branch

@elasticsearchmachine elasticsearchmachine merged commit 9334f34 into elastic:main Jun 17, 2025
24 checks passed
@ywangd ywangd deleted the ES-11328-node-cli-tests branch June 17, 2025 05:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) :Core/Infra/CLI CLI utilities, scripts, and infrastructure Team:Core/Infra Meta label for core/infra team >test Issues or PRs that are addressing/adding tests v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants