Feature: Multi-Project #123336
Merged
Feature: Multi-Project #123336
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merge main into multi-project
As title says. Cluster state recover can happen when master node restart or during an ungraceful fail-over.
Updates the force merge transport action to support multiple projects.
Instead of using an empty global routing table, this PR uses empty project routing table to ensure the project actual has a routing table (albeit empty) when routingTable(projectId) is called.
Merge main into multi-project
…ultiProject-prevalidateRemoval
Updates the service to process data streams in all projects.
On deserialization, state is split into project and cluster based on the namespace, which is hardcoded in `Metadata.Builder.MOVED_PROJECT_RESERVED_STATE`
Explicitly fail the API call if it is in a multi-project setup. Use `ProjectResolver` to get the default (only) project.
Merge main into multi-project
Role Mappings can be stored as a `ProjectCustom` on `ProjectMetadata` (see `RoleMappingMetadata`). This commit ensures that the role mapper looks for the mappings on the active project. A new MultiProject JWT test has been added to cover some of this behaviour
This commit fixes problems that arise from the merge commit c57d41d
This changes the `multi_project.enabled` setting to `test.multi_project.enabled`
An old node writes all tasks in the metadata custom. A new old must be able to read it and separate the cluster-scoped and project-scoped tasks and store them in the right place. This PR does that. Relates: MP-1945, MP-1938
This PR is the follow-up work for MP-1945 and MP-1938 which laid the foundation of two different scoped persistent tasks. It updates the persistent task framework to be aware of the two task types so that it can handle both cluster scope tasks and per-project tasks. Once these changes are in place, we will make health-node to be the first cluster-scope persistent task. Relates: ES-10168
…MP-1965) When the combined tasks diff is used, we need to remove the separate project and cluster tasks from the original diff. Relates: MP-1945, MP-1938
Now that the persistent task framework supports cluster-scoped persistent tasks, we can make the health node persistent task run with cluster scope.
If all we want to know is whether the cluster has any indices, then this new method is more efficient than using the existing "getTotalNumberOfIndices" method
elasticsearchmachine
added
the
serverless-linked
These methods will be removed at some point in the future. By marking them as deprecated, it'll be easier to spot that some code is using these methods. Additionally, this will hopefully prevent people from using these temporary methods.
When a shard recovery fails, we need to able able to handle that properly when there are multiple projects.
We're catching MP exceptions when applying cluster state listeners to avoid noise in the logs, but we shouldn't forget to remove this `catch` block at some point in the future.
This commit reverts the changes to
docs/internal/DistributedArchitectureGuide.md
These changes need to be redone in order to update the code links
so that they point to a more recent version that includes the
`Metadata.ClusterCustom` / `Metadata.ProjectCustom` split
We also need to incorporate documentation updates to reflect the
changes to PersistentTasks.
When #119968 was merged into multi-project we introduced a regression by inserting a call to `.getProject()` within the `RoutingNodes` class that was supposed to be multi-project-aware. This commit replaces those calls with `.indexMetadata` lookups
There is no guarantee that a project has non-null persistent tasks [0]. Null check is already done in most places. This PR adds it in a few more places. [0] Since health-node is now a cluster-scoped persistent task, it has become more likely for the project-scoped tasks to be null. Relates: #123262
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.