Skip to content

[CPS] Search with MRT=true #137701

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Nov 26, 2025
Merged

[CPS] Search with MRT=true #137701

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
ec12300
resolve indices when MRT is true
prwhelan Nov 5, 2025
d732b40
fan out within search action
prwhelan Nov 13, 2025
5b35c5b
Merge branch 'main' into ES-12853
prwhelan Nov 14, 2025
79b7f66
checkstyle lol
prwhelan Nov 14, 2025
161ca5e
Add unit tests
prwhelan Nov 17, 2025
95fee97
move back to coordinator thread on complete
prwhelan Nov 19, 2025
6255768
Merge branch 'main' into ES-12853
prwhelan Nov 19, 2025
bc617da
Revert "Add unit tests"
prwhelan Nov 19, 2025
497e80a
remove local resolution
prwhelan Nov 24, 2025
5e721da
more docs, more tests
prwhelan Nov 25, 2025
6a524c4
Merge branch 'main' into ES-12853
prwhelan Nov 25, 2025
ca8dd02
Merge branch 'main' into ES-12853
prwhelan Nov 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion server/src/internalClusterTest/java/org/elasticsearch/action/search/TransportSearchIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,7 @@ public void testLocalClusterAlias() throws ExecutionException, InterruptedExcept
parentTaskId,
new SearchRequest(),
Strings.EMPTY_ARRAY,
SearchRequest.DEFAULT_INDICES_OPTIONS,
"local",
nowInMillis,
randomBoolean()
Expand All @@ -158,6 +159,7 @@ public void testLocalClusterAlias() throws ExecutionException, InterruptedExcept
parentTaskId,
new SearchRequest(),
Strings.EMPTY_ARRAY,
SearchRequest.DEFAULT_INDICES_OPTIONS,
"",
nowInMillis,
randomBoolean()
Expand Down Expand Up @@ -205,6 +207,7 @@ public void testAbsoluteStartMillis() throws ExecutionException, InterruptedExce
parentTaskId,
new SearchRequest(),
Strings.EMPTY_ARRAY,
SearchRequest.DEFAULT_INDICES_OPTIONS,
"",
0,
randomBoolean()
Expand All @@ -216,6 +219,7 @@ public void testAbsoluteStartMillis() throws ExecutionException, InterruptedExce
parentTaskId,
new SearchRequest(),
Strings.EMPTY_ARRAY,
SearchRequest.DEFAULT_INDICES_OPTIONS,
"",
0,
randomBoolean()
Expand All @@ -231,6 +235,7 @@ public void testAbsoluteStartMillis() throws ExecutionException, InterruptedExce
parentTaskId,
new SearchRequest(),
Strings.EMPTY_ARRAY,
SearchRequest.DEFAULT_INDICES_OPTIONS,
"",
0,
randomBoolean()
Expand Down Expand Up @@ -279,7 +284,15 @@ public void testFinalReduce() throws ExecutionException, InterruptedException {
{
SearchRequest searchRequest = randomBoolean()
? originalRequest
: SearchRequest.subSearchRequest(taskId, originalRequest, Strings.EMPTY_ARRAY, "remote", nowInMillis, true);
: SearchRequest.subSearchRequest(
taskId,
originalRequest,
Strings.EMPTY_ARRAY,
originalRequest.indicesOptions(),
"remote",
nowInMillis,
true
);
assertResponse(client().search(searchRequest), searchResponse -> {
assertEquals(2, searchResponse.getHits().getTotalHits().value());
InternalAggregations aggregations = searchResponse.getAggregations();
Expand All @@ -292,6 +305,7 @@ public void testFinalReduce() throws ExecutionException, InterruptedException {
taskId,
originalRequest,
Strings.EMPTY_ARRAY,
originalRequest.indicesOptions(),
"remote",
nowInMillis,
false
Expand Down
29 changes: 29 additions & 0 deletions server/src/main/java/org/elasticsearch/action/ResolvedIndices.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,35 @@ public static ResolvedIndices resolveWithPIT(
);
}

/**
* Create a new {@link ResolvedIndices} instance from a Map of Projects to {@link ResolvedIndexExpressions}.
* This method guarantees that the resulting remote project contains at least one index resolved for the mapped
Copy link
Contributor

@quux00 quux00 Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like I'm missing something. This statement:

This method guarantees that the resulting remote project contains at least one index resolved for the mapped ResolvedIndexExpressions

doesn't seem right to me. The check that guarantees this is the call to CrossProjectIndexResolutionValidator.validate in mergeResolvedIndices, isn't it?

Copy link
Contributor

@pawankartik-elastic pawankartik-elastic Nov 24, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a guarantee on the map's (remoteIndices) structuring rather than a behaviour ascertaining. Either way, my opinion is that it's not immediately clear what the doc means, unless you're by default familiar with the CPS specifics. Could this be rewritten or clarified?

Copy link
Member Author

@prwhelan prwhelan Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 268 through 280 iterates over a map of ResolvedIndexExpressions and filters the internal list of ResolvedIndexExpression for entities that have .localIndexResolutionResult() == SUCCESS and at least one index.

For example, if project-1 had the original index expression logs* map to indices ["logs-1", "logs-2"], then the result has project-1 map to ["logs*"]. If project-1 had logs* map to [], then project-1 is not included in the resulting map.

So, each remote project that is keyed in the resulting ResolvedIndices has at least one index that is mapped from the original index expression.

The thing I would like to verify is the assumption that we made in the slack thread, where we set the remote project's index in the OriginalIndices to be the original index expression and not the resolved index (e.g. ["logs*"] and not ["logs-1", "logs-2"].

Copy link
Contributor

@pawankartik-elastic pawankartik-elastic Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing I would like to verify is the assumption that we made in the slack thread, where we set the remote project's index in the OriginalIndices to be the original index expression and not the resolved index

That's right. We want to associate with the user input index expressions and not the resolved values. You're on the right track.

* {@link ResolvedIndexExpressions}. The resulting {@link ResolvedIndices#getRemoteClusterIndices()} will map to the original index
* expression provided in {@link ResolvedIndexExpression#original()}.
*/
public static ResolvedIndices resolveWithIndexExpressions(
OriginalIndices localIndices,
Map<Index, IndexMetadata> localIndexMetadata,
Map<String, ResolvedIndexExpressions> remoteExpressions,
IndicesOptions indicesOptions
) {
Map<String, OriginalIndices> remoteIndices = remoteExpressions.entrySet().stream().collect(HashMap::new, (map, entry) -> {
var indices = entry.getValue().expressions().stream().filter(expression -> {
var resolvedExpressions = expression.localExpressions();
var successfulResolution = resolvedExpressions
.localIndexResolutionResult() == ResolvedIndexExpression.LocalIndexResolutionResult.SUCCESS;
// if the expression is a wildcard, it will be successful even if there are no indices, so filter for no indices
var hasResolvedIndices = resolvedExpressions.indices().isEmpty() == false;
return successfulResolution && hasResolvedIndices;
}).map(ResolvedIndexExpression::original).toArray(String[]::new);
if (indices.length > 0) {
map.put(entry.getKey(), new OriginalIndices(indices, indicesOptions));
}
}, Map::putAll);

return new ResolvedIndices(remoteIndices, localIndices, localIndexMetadata);
}

private static Map<Index, IndexMetadata> resolveLocalIndexMetadata(
Index[] concreteLocalIndices,
ProjectMetadata projectMetadata,
Expand Down
7 changes: 5 additions & 2 deletions server/src/main/java/org/elasticsearch/action/search/SearchRequest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ public void setProjectRouting(@Nullable String projectRouting) {
* @param parentTaskId the parent taskId of the original search request
* @param originalSearchRequest the original search request
* @param indices the indices to search against
* @param indicesOptions the indicesOptions to search with
* @param clusterAlias the alias to prefix index names with in the returned search results
* @param absoluteStartMillis the absolute start time to be used on the remote clusters to ensure that the same value is used
* @param finalReduce whether the reduction should be final or not
Expand All @@ -204,6 +205,7 @@ static SearchRequest subSearchRequest(
TaskId parentTaskId,
SearchRequest originalSearchRequest,
String[] indices,
IndicesOptions indicesOptions,
String clusterAlias,
long absoluteStartMillis,
boolean finalReduce
Expand All @@ -217,6 +219,7 @@ static SearchRequest subSearchRequest(
}
final SearchRequest request = new SearchRequest(originalSearchRequest, indices, clusterAlias, absoluteStartMillis, finalReduce);
request.setParentTask(parentTaskId);
request.indicesOptions(indicesOptions);
return request;
}

Expand Down Expand Up @@ -414,8 +417,8 @@ boolean isFinalReduce() {
/**
* Returns the current time in milliseconds from the time epoch, to be used for the execution of this search request. Used to
* ensure that the same value, determined by the coordinating node, is used on all nodes involved in the execution of the search
* request. When created through {@link #subSearchRequest(TaskId, SearchRequest, String[], String, long, boolean)}, this method returns
* the provided current time, otherwise it will return {@link System#currentTimeMillis()}.
* request. When created through {@link #subSearchRequest(TaskId, SearchRequest, String[], IndicesOptions, String, long, boolean)},
* this method returns the provided current time, otherwise it will return {@link System#currentTimeMillis()}.
*/
long getOrCreateAbsoluteStartMillis() {
return absoluteStartMillis == DEFAULT_ABSOLUTE_START_MILLIS ? System.currentTimeMillis() : absoluteStartMillis;
Expand Down
Loading