Skip to content

Make deprecation plugin project-aware #130364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 1, 2025
Merged

Make deprecation plugin project-aware #130364

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3dae703
Make deprecation plugin project-aware
nielsbauman Jun 30, 2025
7e6af7b
Merge branch 'main' into deprecation-mp
nielsbauman Jul 1, 2025
046cb01
Remove `CHECKS` abstraction
nielsbauman Jul 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 5 additions & 12 deletions ...recation/src/main/java/org/elasticsearch/xpack/deprecation/ClusterDeprecationChecker.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,42 +9,35 @@

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.common.TriConsumer;
import org.elasticsearch.xcontent.NamedXContentRegistry;
import org.elasticsearch.xpack.core.deprecation.DeprecationIssue;
import org.elasticsearch.xpack.core.transform.transforms.TransformConfig;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.function.BiConsumer;

/**
* Cluster-specific deprecation checks, this is used to populate the {@code cluster_settings} field
*/
public class ClusterDeprecationChecker {

private static final Logger logger = LogManager.getLogger(ClusterDeprecationChecker.class);
private final List<TriConsumer<ClusterState, List<TransformConfig>, List<DeprecationIssue>>> CHECKS = List.of(
this::checkTransformSettings
);
private final List<BiConsumer<List<TransformConfig>, List<DeprecationIssue>>> CHECKS = List.of(this::checkTransformSettings);
private final NamedXContentRegistry xContentRegistry;

ClusterDeprecationChecker(NamedXContentRegistry xContentRegistry) {
this.xContentRegistry = xContentRegistry;
}

public List<DeprecationIssue> check(ClusterState clusterState, List<TransformConfig> transformConfigs) {
public List<DeprecationIssue> check(List<TransformConfig> transformConfigs) {
List<DeprecationIssue> allIssues = new ArrayList<>();
CHECKS.forEach(check -> check.apply(clusterState, transformConfigs, allIssues));
CHECKS.forEach(check -> check.accept(transformConfigs, allIssues));
Copy link
Member

@PeteGillinElastic PeteGillinElastic Jul 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we feel like the CHECKS abstraction is still pulling its weight here? There's one check hard-coded, we could just call checkTransformSettings directly, and it'd be more readable. If we needed to add more checks in the future, it's not clear to me that adding a method to the list would be better than adding another method call here. Plus, it's not clear to me that the BiConsumer would have the correct signature for a theoretical future check, either. (Presumably, at some point in the past there was a check which consumed ClusterState, hence the redundant parameter. A future check might need that, or might need the ProjectMetadata, but we can't really guess. now.)

Copy link
Contributor Author

@nielsbauman nielsbauman Jul 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hear what you're saying. My intuition was that this CHECKS field makes this class more consistent with the other deprecation checkers. But I don't have strong feelings, so I'm fine with calling that method directly.

Copy link
Contributor Author

@nielsbauman nielsbauman Jul 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I think I agree with you that the CHECKS abstraction isn't super valuable, and that goes for the other classes too. I don't think it's worth opening a PR for those other classes, but perhaps I'll remember it if I ever need to touch this code again.

return allIssues;
}

private void checkTransformSettings(
ClusterState clusterState,
List<TransformConfig> transformConfigs,
List<DeprecationIssue> allIssues
) {
private void checkTransformSettings(List<TransformConfig> transformConfigs, List<DeprecationIssue> allIssues) {
for (var config : transformConfigs) {
try {
allIssues.addAll(config.checkForDeprecations(xContentRegistry));
Expand Down
34 changes: 16 additions & 18 deletions ...ation/src/main/java/org/elasticsearch/xpack/deprecation/DataStreamDeprecationChecker.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@

import org.elasticsearch.Version;
import org.elasticsearch.action.support.IndicesOptions;
import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.cluster.metadata.DataStream;
import org.elasticsearch.cluster.metadata.IndexNameExpressionResolver;
import org.elasticsearch.cluster.metadata.ProjectMetadata;
import org.elasticsearch.index.Index;
import org.elasticsearch.xpack.core.deprecation.DeprecatedIndexPredicate;
import org.elasticsearch.xpack.core.deprecation.DeprecationIssue;
Expand All @@ -33,7 +33,7 @@
public class DataStreamDeprecationChecker implements ResourceDeprecationChecker {

public static final String NAME = "data_streams";
private static final List<BiFunction<DataStream, ClusterState, DeprecationIssue>> DATA_STREAM_CHECKS = List.of(
private static final List<BiFunction<DataStream, ProjectMetadata, DeprecationIssue>> DATA_STREAM_CHECKS = List.of(
DataStreamDeprecationChecker::oldIndicesCheck,
DataStreamDeprecationChecker::ignoredOldIndicesCheck
);
Expand All @@ -44,38 +44,38 @@ public DataStreamDeprecationChecker(IndexNameExpressionResolver indexNameExpress
}

/**
* @param clusterState The cluster state provided for the checker
* @param project The project metadata provided for the checker
* @param request not used yet in these checks
* @param precomputedData not used yet in these checks
* @return the name of the data streams that have violated the checks with their respective warnings.
*/
@Override
public Map<String, List<DeprecationIssue>> check(
ClusterState clusterState,
ProjectMetadata project,
DeprecationInfoAction.Request request,
TransportDeprecationInfoAction.PrecomputedData precomputedData
) {
return check(clusterState);
return check(project);
}

/**
* @param clusterState The cluster state provided for the checker
* @param project The project metadata provided for the checker
* @return the name of the data streams that have violated the checks with their respective warnings.
*/
public Map<String, List<DeprecationIssue>> check(ClusterState clusterState) {
public Map<String, List<DeprecationIssue>> check(ProjectMetadata project) {
List<String> dataStreamNames = indexNameExpressionResolver.dataStreamNames(
clusterState,
project,
IndicesOptions.LENIENT_EXPAND_OPEN_CLOSED_HIDDEN
);
if (dataStreamNames.isEmpty()) {
return Map.of();
}
Map<String, List<DeprecationIssue>> dataStreamIssues = new HashMap<>();
for (String dataStreamName : dataStreamNames) {
DataStream dataStream = clusterState.metadata().getProject().dataStreams().get(dataStreamName);
DataStream dataStream = project.dataStreams().get(dataStreamName);
if (dataStream.isSystem() == false) {
List<DeprecationIssue> issuesForSingleDataStream = DATA_STREAM_CHECKS.stream()
.map(c -> c.apply(dataStream, clusterState))
.map(c -> c.apply(dataStream, project))
.filter(Objects::nonNull)
.toList();
if (issuesForSingleDataStream.isEmpty() == false) {
Expand All @@ -86,10 +86,10 @@ public Map<String, List<DeprecationIssue>> check(ClusterState clusterState) {
return dataStreamIssues.isEmpty() ? Map.of() : dataStreamIssues;
}

static DeprecationIssue oldIndicesCheck(DataStream dataStream, ClusterState clusterState) {
static DeprecationIssue oldIndicesCheck(DataStream dataStream, ProjectMetadata project) {
List<Index> backingIndices = dataStream.getIndices();

Set<String> indicesNeedingUpgrade = getReindexRequiredIndices(backingIndices, clusterState, false);
Set<String> indicesNeedingUpgrade = getReindexRequiredIndices(backingIndices, project, false);

if (indicesNeedingUpgrade.isEmpty() == false) {
return new DeprecationIssue(
Expand All @@ -110,9 +110,9 @@ static DeprecationIssue oldIndicesCheck(DataStream dataStream, ClusterState clus
return null;
}

static DeprecationIssue ignoredOldIndicesCheck(DataStream dataStream, ClusterState clusterState) {
static DeprecationIssue ignoredOldIndicesCheck(DataStream dataStream, ProjectMetadata project) {
List<Index> backingIndices = dataStream.getIndices();
Set<String> ignoredIndices = getReindexRequiredIndices(backingIndices, clusterState, true);
Set<String> ignoredIndices = getReindexRequiredIndices(backingIndices, project, true);
if (ignoredIndices.isEmpty() == false) {
return new DeprecationIssue(
DeprecationIssue.Level.WARNING,
Expand All @@ -135,13 +135,11 @@ static DeprecationIssue ignoredOldIndicesCheck(DataStream dataStream, ClusterSta

private static Set<String> getReindexRequiredIndices(
List<Index> backingIndices,
ClusterState clusterState,
ProjectMetadata project,
boolean filterToBlockedStatus
) {
return backingIndices.stream()
.filter(
DeprecatedIndexPredicate.getReindexRequiredPredicate(clusterState.metadata().getProject(), filterToBlockedStatus, false)
)
.filter(DeprecatedIndexPredicate.getReindexRequiredPredicate(project, filterToBlockedStatus, false))
.map(Index::getName)
.collect(Collectors.toUnmodifiableSet());
}
Expand Down
14 changes: 7 additions & 7 deletions ...cation/src/main/java/org/elasticsearch/xpack/deprecation/IlmPolicyDeprecationChecker.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

package org.elasticsearch.xpack.deprecation;

import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.cluster.metadata.ProjectMetadata;
import org.elasticsearch.xpack.core.deprecation.DeprecationIssue;
import org.elasticsearch.xpack.core.ilm.AllocateAction;
import org.elasticsearch.xpack.core.ilm.FreezeAction;
Expand Down Expand Up @@ -36,26 +36,26 @@ public class IlmPolicyDeprecationChecker implements ResourceDeprecationChecker {
private final List<Function<LifecyclePolicy, DeprecationIssue>> checks = List.of(this::checkLegacyTiers, this::checkFrozenAction);

/**
* @param clusterState The cluster state provided for the checker
* @param project The project metadata provided for the checker
* @param request not used yet in these checks
* @param precomputedData not used yet in these checks
* @return the name of the data streams that have violated the checks with their respective warnings.
*/
@Override
public Map<String, List<DeprecationIssue>> check(
ClusterState clusterState,
ProjectMetadata project,
DeprecationInfoAction.Request request,
TransportDeprecationInfoAction.PrecomputedData precomputedData
) {
return check(clusterState);
return check(project);
}

/**
* @param clusterState The cluster state provided for the checker
* @param project The project metadata provided for the checker
* @return the name of the data streams that have violated the checks with their respective warnings.
*/
Map<String, List<DeprecationIssue>> check(ClusterState clusterState) {
IndexLifecycleMetadata lifecycleMetadata = clusterState.metadata().getProject().custom(IndexLifecycleMetadata.TYPE);
Map<String, List<DeprecationIssue>> check(ProjectMetadata project) {
IndexLifecycleMetadata lifecycleMetadata = project.custom(IndexLifecycleMetadata.TYPE);
if (lifecycleMetadata == null || lifecycleMetadata.getPolicyMetadatas().isEmpty()) {
return Map.of();
}
Expand Down
34 changes: 17 additions & 17 deletions ...eprecation/src/main/java/org/elasticsearch/xpack/deprecation/IndexDeprecationChecker.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@
package org.elasticsearch.xpack.deprecation;

import org.elasticsearch.Version;
import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.cluster.metadata.IndexMetadata;
import org.elasticsearch.cluster.metadata.IndexNameExpressionResolver;
import org.elasticsearch.cluster.metadata.MappingMetadata;
import org.elasticsearch.cluster.metadata.ProjectMetadata;
import org.elasticsearch.common.TriFunction;
import org.elasticsearch.common.time.DateFormatter;
import org.elasticsearch.common.time.LegacyFormatNames;
Expand Down Expand Up @@ -44,7 +44,7 @@ public class IndexDeprecationChecker implements ResourceDeprecationChecker {
public static final String NAME = "index_settings";

private final IndexNameExpressionResolver indexNameExpressionResolver;
private final List<TriFunction<IndexMetadata, ClusterState, Map<String, List<String>>, DeprecationIssue>> checks = List.of(
private final List<TriFunction<IndexMetadata, ProjectMetadata, Map<String, List<String>>, DeprecationIssue>> checks = List.of(
this::oldIndicesCheck,
this::ignoredOldIndicesCheck,
this::translogRetentionSettingCheck,
Expand All @@ -60,17 +60,17 @@ public IndexDeprecationChecker(IndexNameExpressionResolver indexNameExpressionRe

@Override
public Map<String, List<DeprecationIssue>> check(
ClusterState clusterState,
ProjectMetadata project,
DeprecationInfoAction.Request request,
TransportDeprecationInfoAction.PrecomputedData precomputedData
) {
Map<String, List<DeprecationIssue>> indexSettingsIssues = new HashMap<>();
String[] concreteIndexNames = indexNameExpressionResolver.concreteIndexNames(clusterState, request);
String[] concreteIndexNames = indexNameExpressionResolver.concreteIndexNames(project, request);
Map<String, List<String>> indexToTransformIds = indexToTransformIds(precomputedData.transformConfigs());
for (String concreteIndex : concreteIndexNames) {
IndexMetadata indexMetadata = clusterState.getMetadata().getProject().index(concreteIndex);
IndexMetadata indexMetadata = project.index(concreteIndex);
List<DeprecationIssue> singleIndexIssues = checks.stream()
.map(c -> c.apply(indexMetadata, clusterState, indexToTransformIds))
.map(c -> c.apply(indexMetadata, project, indexToTransformIds))
.filter(Objects::nonNull)
.toList();
if (singleIndexIssues.isEmpty() == false) {
Expand All @@ -90,13 +90,13 @@ public String getName() {

private DeprecationIssue oldIndicesCheck(
IndexMetadata indexMetadata,
ClusterState clusterState,
ProjectMetadata project,
Map<String, List<String>> indexToTransformIds
) {
// TODO: this check needs to be revised. It's trivially true right now.
IndexVersion currentCompatibilityVersion = indexMetadata.getCompatibilityVersion();
// We intentionally exclude indices that are in data streams because they will be picked up by DataStreamDeprecationChecks
if (DeprecatedIndexPredicate.reindexRequired(indexMetadata, false, false) && isNotDataStreamIndex(indexMetadata, clusterState)) {
if (DeprecatedIndexPredicate.reindexRequired(indexMetadata, false, false) && isNotDataStreamIndex(indexMetadata, project)) {
var transforms = transformIdsForIndex(indexMetadata, indexToTransformIds);
if (transforms.isEmpty() == false) {
return new DeprecationIssue(
Expand Down Expand Up @@ -134,12 +134,12 @@ private List<String> transformIdsForIndex(IndexMetadata indexMetadata, Map<Strin

private DeprecationIssue ignoredOldIndicesCheck(
IndexMetadata indexMetadata,
ClusterState clusterState,
ProjectMetadata project,
Map<String, List<String>> indexToTransformIds
) {
IndexVersion currentCompatibilityVersion = indexMetadata.getCompatibilityVersion();
// We intentionally exclude indices that are in data streams because they will be picked up by DataStreamDeprecationChecks
if (DeprecatedIndexPredicate.reindexRequired(indexMetadata, true, false) && isNotDataStreamIndex(indexMetadata, clusterState)) {
if (DeprecatedIndexPredicate.reindexRequired(indexMetadata, true, false) && isNotDataStreamIndex(indexMetadata, project)) {
var transforms = transformIdsForIndex(indexMetadata, indexToTransformIds);
if (transforms.isEmpty() == false) {
return new DeprecationIssue(
Expand Down Expand Up @@ -175,13 +175,13 @@ private DeprecationIssue ignoredOldIndicesCheck(
return null;
}

private boolean isNotDataStreamIndex(IndexMetadata indexMetadata, ClusterState clusterState) {
return clusterState.metadata().getProject().findDataStreams(indexMetadata.getIndex().getName()).isEmpty();
private boolean isNotDataStreamIndex(IndexMetadata indexMetadata, ProjectMetadata project) {
return project.findDataStreams(indexMetadata.getIndex().getName()).isEmpty();
}

private DeprecationIssue translogRetentionSettingCheck(
IndexMetadata indexMetadata,
ClusterState clusterState,
ProjectMetadata project,
Map<String, List<String>> ignored
) {
final boolean softDeletesEnabled = IndexSettings.INDEX_SOFT_DELETES_SETTING.get(indexMetadata.getSettings());
Expand Down Expand Up @@ -210,7 +210,7 @@ private DeprecationIssue translogRetentionSettingCheck(
return null;
}

private DeprecationIssue checkIndexDataPath(IndexMetadata indexMetadata, ClusterState clusterState, Map<String, List<String>> ignored) {
private DeprecationIssue checkIndexDataPath(IndexMetadata indexMetadata, ProjectMetadata project, Map<String, List<String>> ignored) {
if (IndexMetadata.INDEX_DATA_PATH_SETTING.exists(indexMetadata.getSettings())) {
final String message = String.format(
Locale.ROOT,
Expand All @@ -226,7 +226,7 @@ private DeprecationIssue checkIndexDataPath(IndexMetadata indexMetadata, Cluster

private DeprecationIssue storeTypeSettingCheck(
IndexMetadata indexMetadata,
ClusterState clusterState,
ProjectMetadata project,
Map<String, List<String>> ignored
) {
final String storeType = IndexModule.INDEX_STORE_TYPE_SETTING.get(indexMetadata.getSettings());
Expand All @@ -247,7 +247,7 @@ private DeprecationIssue storeTypeSettingCheck(

private DeprecationIssue legacyRoutingSettingCheck(
IndexMetadata indexMetadata,
ClusterState clusterState,
ProjectMetadata project,
Map<String, List<String>> ignored
) {
List<String> deprecatedSettings = LegacyTiersDetection.getDeprecatedFilteredAllocationSettings(indexMetadata.getSettings());
Expand Down Expand Up @@ -339,7 +339,7 @@ private List<String> findInPropertiesRecursively(

private DeprecationIssue deprecatedCamelCasePattern(
IndexMetadata indexMetadata,
ClusterState clusterState,
ProjectMetadata project,
Map<String, List<String>> ignored
) {
List<String> fields = new ArrayList<>();
Expand Down
6 changes: 3 additions & 3 deletions ...ecation/src/main/java/org/elasticsearch/xpack/deprecation/ResourceDeprecationChecker.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

package org.elasticsearch.xpack.deprecation;

import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.cluster.metadata.ProjectMetadata;
import org.elasticsearch.xpack.core.deprecation.DeprecationIssue;

import java.util.List;
Expand All @@ -23,12 +23,12 @@ public interface ResourceDeprecationChecker {
/**
* This runs the checks for the current deprecation checker.
*
* @param clusterState The cluster state provided for the checker
* @param project The project metadata provided for the checker
* @param request The deprecation request that triggered this check
* @param precomputedData Data that have been remotely retrieved and might be useful in the checks
*/
Map<String, List<DeprecationIssue>> check(
ClusterState clusterState,
ProjectMetadata project,
DeprecationInfoAction.Request request,
TransportDeprecationInfoAction.PrecomputedData precomputedData
);
Expand Down
16 changes: 8 additions & 8 deletions ...ecation/src/main/java/org/elasticsearch/xpack/deprecation/TemplateDeprecationChecker.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@

package org.elasticsearch.xpack.deprecation;

import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.cluster.metadata.ComponentTemplate;
import org.elasticsearch.cluster.metadata.ComposableIndexTemplate;
import org.elasticsearch.cluster.metadata.ProjectMetadata;
import org.elasticsearch.cluster.metadata.Template;
import org.elasticsearch.common.xcontent.XContentHelper;
import org.elasticsearch.index.mapper.SourceFieldMapper;
Expand Down Expand Up @@ -41,27 +41,27 @@ public class TemplateDeprecationChecker implements ResourceDeprecationChecker {
);

/**
* @param clusterState The cluster state provided for the checker
* @param project The project metadata provided for the checker
* @param request not used yet in these checks
* @param precomputedData not used yet in these checks
* @return the name of the data streams that have violated the checks with their respective warnings.
*/
@Override
public Map<String, List<DeprecationIssue>> check(
ClusterState clusterState,
ProjectMetadata project,
DeprecationInfoAction.Request request,
TransportDeprecationInfoAction.PrecomputedData precomputedData
) {
return check(clusterState);
return check(project);
}

/**
* @param clusterState The cluster state provided for the checker
* @param project The project metadata provided for the checker
* @return the name of the data streams that have violated the checks with their respective warnings.
*/
Map<String, List<DeprecationIssue>> check(ClusterState clusterState) {
var indexTemplates = clusterState.metadata().getProject().templatesV2().entrySet();
var componentTemplates = clusterState.metadata().getProject().componentTemplates().entrySet();
Map<String, List<DeprecationIssue>> check(ProjectMetadata project) {
var indexTemplates = project.templatesV2().entrySet();
var componentTemplates = project.componentTemplates().entrySet();
if (indexTemplates.isEmpty() && componentTemplates.isEmpty()) {
return Map.of();
}
Expand Down
Loading