Skip to content

Conversation

@alexey-ivanov-es
Copy link
Contributor

@alexey-ivanov-es alexey-ivanov-es commented Jun 6, 2025

To better support project restoration after deletion, this change moves project Settings from ProjectMetadata to the new custom in the ClusterState. It also introduces a new transport version for cluster state serialization. Reserved cluster state for project settings remains within ProjectMetadata.

Note: In mixed-version multiproject clusters, this may cause existing settings for projects to temporarily disappear until all nodes have been upgraded and restarted.

@elasticsearchmachine elasticsearchmachine added serverless-linked Added by automation, don't add manually v9.1.0 labels Jun 6, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @alexey-ivanov-es, I've created a changelog YAML for you.

@alexey-ivanov-es alexey-ivanov-es marked this pull request as ready for review June 6, 2025 14:28
@alexey-ivanov-es alexey-ivanov-es requested a review from a team as a code owner June 6, 2025 14:28
@elasticsearchmachine elasticsearchmachine added the Team:Core/Infra Meta label for core/infra team label Jun 6, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

public static final TransportVersion ILM_ADD_SKIP_SETTING = def(9_089_0_00);
public static final TransportVersion ML_INFERENCE_MISTRAL_CHAT_COMPLETION_ADDED = def(9_090_0_00);
public static final TransportVersion IDP_CUSTOM_SAML_ATTRIBUTES_ALLOW_LIST = def(9_091_0_00);
public static final TransportVersion CLUSTER_STATE_PROJECTS_SETTINGS = def(9_092_0_00);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we actually need to introduce a new transport version here? The per-project stuff was never included in a stack release, and even in serverless it's behind the multi-project flag which isn't enabled in any production environment. I believe it's safe to remove this w/o introducing a new transport version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rjernst is my logic sound here?

Copy link
Member

@rjernst rjernst Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe. I think the problem is when the feature flag is removed, a transport version would need to be introduced. Adding it here seems safer.

}

private static ClusterState buildMultiProjectClusterState(DiscoveryNode... nodes) {
ProjectId projectId1 = ProjectId.fromId("3LftaL7hgfXAsF60Gm6jcD");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use randomProjectIdOrDefault() or randomUniqueProjectId() here instead of hard-coding values.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This project-id is used in the serialized form of cluster state - using a random id would require generating the expected string, and I don't think it's worth it

Copy link
Contributor

@mark-vieira mark-vieira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor nits but otherwise LGTM.

@mark-vieira
Copy link
Contributor

Changes LGTM.

@alexey-ivanov-es alexey-ivanov-es merged commit 876c456 into elastic:main Jun 24, 2025
33 checks passed
mridula-s109 pushed a commit to mridula-s109/elasticsearch that referenced this pull request Jun 25, 2025
To better support project restoration after deletion, this change moves project Settings from ProjectMetadata to the new custom in the ClusterState. It also introduces a new transport version for cluster state serialization. Reserved cluster state for project settings remains within ProjectMetadata.

Note: In mixed-version multiproject clusters, this may cause existing settings for projects to temporarily disappear until all nodes have been upgraded and restarted.
@alexey-ivanov-es alexey-ivanov-es deleted the ES-11934 branch August 29, 2025 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Core/Infra/Settings Settings infrastructure and APIs >non-issue serverless-linked Added by automation, don't add manually Team:Core/Infra Meta label for core/infra team v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants