Skip to content

ccs_minimize_roundtrips should default to true for async search for CPS #135614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Oct 3, 2025
Merged

ccs_minimize_roundtrips should default to true for async search for CPS #135614

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
972f34e
`ccs_minimize_roundtrips` should default to `true` for async search f…
pawankartik-elastic Sep 29, 2025
a72a5d9
Merge branch 'main' into pkar/cps-mrt-default-true
pawankartik-elastic Sep 29, 2025
10662e6
Merge branch 'main' into pkar/cps-mrt-default-true
pawankartik-elastic Sep 29, 2025
80461af
Set `MRT` when parsing instead of waiting for `TSA`
pawankartik-elastic Sep 30, 2025
2b3117d
Merge branch 'main' into pkar/cps-mrt-default-true
pawankartik-elastic Sep 30, 2025
6ae750a
Merge branch 'main' into pkar/cps-mrt-default-true
pawankartik-elastic Sep 30, 2025
f35bf00
Address some review comments
pawankartik-elastic Oct 1, 2025
264302e
Add comments to endpoints clarifying CPS context argument
pawankartik-elastic Oct 1, 2025
fcf4f09
Propagate CPS context value
pawankartik-elastic Oct 1, 2025
fbffdfe
Add clarifying comments for CPS context argument
pawankartik-elastic Oct 1, 2025
d2306a4
Merge branch 'main' into pkar/cps-mrt-default-true
pawankartik-elastic Oct 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions ...ng-mustache/src/main/java/org/elasticsearch/script/mustache/RestSearchTemplateAction.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@

import java.io.IOException;
import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.function.Predicate;

Expand All @@ -38,10 +39,12 @@ public class RestSearchTemplateAction extends BaseRestHandler {

private final Predicate<NodeFeature> clusterSupportsFeature;
private final Settings settings;
private final boolean inCpsContext;

public RestSearchTemplateAction(Predicate<NodeFeature> clusterSupportsFeature, Settings settings) {
this.clusterSupportsFeature = clusterSupportsFeature;
this.settings = settings;
this.inCpsContext = settings != null && settings.getAsBoolean("serverless.cross_project.enabled", false);
}

@Override
Expand All @@ -61,7 +64,7 @@ public String getName() {

@Override
public RestChannelConsumer prepareRequest(RestRequest request, NodeClient client) throws IOException {
if (settings != null && settings.getAsBoolean("serverless.cross_project.enabled", false)) {
if (inCpsContext) {
// accept but drop project_routing param until fully supported
request.param("project_routing");
}
Expand All @@ -73,7 +76,9 @@ public RestChannelConsumer prepareRequest(RestRequest request, NodeClient client
request,
null,
clusterSupportsFeature,
size -> searchRequest.source().size(size)
size -> searchRequest.source().size(size),
// This endpoint is CPS-enabled so propagate the right value.
Optional.of(inCpsContext)
);

// Creates the search template request
Expand Down
10 changes: 9 additions & 1 deletion modules/reindex/src/main/java/org/elasticsearch/reindex/AbstractBulkByQueryRestHandler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@

import java.io.IOException;
import java.util.Map;
import java.util.Optional;
import java.util.function.Consumer;
import java.util.function.Predicate;

Expand All @@ -52,7 +53,14 @@ protected void parseInternalRequest(
SearchRequest searchRequest = internal.getSearchRequest();

try (XContentParser parser = extractRequestSpecificFields(restRequest, bodyConsumers)) {
RestSearchAction.parseSearchRequest(searchRequest, restRequest, parser, clusterSupportsFeature, size -> failOnSizeSpecified());
RestSearchAction.parseSearchRequest(
searchRequest,
restRequest,
parser,
clusterSupportsFeature,
size -> failOnSizeSpecified(),
Optional.empty()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know that none of the subclasses of this class are cross-project enabled? We'll need to track those down and ensure that this Optional.empty() is valid for all. Otherwise, that arg will need to be added to the parseInternalRequest method signature.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Irrespective of whether they're CPS-enabled or not, Optional.empty() could be a valid argument since:

  1. Their behaviour does not change for this value, and,
  2. This value can be a placeholder to denote that an endpoint does not need any CPS-specific handling during parsing.

Passing in true or false would've marked their behaviour more explicitly. I'd prefer evaluating their CPS compatibility and updating this value in a separate PR. Wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I follow. If any CPS-enabled endpoints are using this AbstractBulkByQueryRestHandler class as a parent, then they must not pass in Optional.empty() because setting ccs_minimize_roundtrips is not allowed in that case. I doubt any bulk endpoints are CCS enabled, but we need to track them down to make sure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two endpoints using AbstractBulkByQueryRestHandler are _update_by_query and _delete_by_query. They accept CCS patterns such as foo*,remote*:bar*. However, the remotes included are not affected. For example, /*,*:*/_delete_by_query does not affect the remotes' docs. Instead, I see weird errors. If I specify a single remote, I no longer get any errors, but the docs still don't get deleted. I'm assuming this is an oversight, and that these endpoints don't play well with Cross Cluster ops in the first place. In that case, Optional.empty() is the right value here.

Copy link
Contributor Author

@pawankartik-elastic pawankartik-elastic Oct 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, we should prevent Cross Cluster ops for them like we're planning to do for Fleet searches.
Edit: I just found tickets that track this internally.

);
}

searchRequest.source().size(restRequest.paramAsInt("scroll_size", searchRequest.source().size()));
Expand Down
52 changes: 44 additions & 8 deletions server/src/main/java/org/elasticsearch/rest/action/search/RestSearchAction.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
import java.util.Arrays;
import java.util.List;
import java.util.Locale;
import java.util.Optional;
import java.util.Set;
import java.util.function.IntConsumer;
import java.util.function.Predicate;
Expand All @@ -68,6 +69,7 @@ public class RestSearchAction extends BaseRestHandler {
private final SearchUsageHolder searchUsageHolder;
private final Predicate<NodeFeature> clusterSupportsFeature;
private final Settings settings;
private final boolean inCpsContext;

public RestSearchAction(SearchUsageHolder searchUsageHolder, Predicate<NodeFeature> clusterSupportsFeature) {
this(searchUsageHolder, clusterSupportsFeature, null);
Expand All @@ -77,6 +79,7 @@ public RestSearchAction(SearchUsageHolder searchUsageHolder, Predicate<NodeFeatu
this.searchUsageHolder = searchUsageHolder;
this.clusterSupportsFeature = clusterSupportsFeature;
this.settings = settings;
this.inCpsContext = settings != null && settings.getAsBoolean("serverless.cross_project.enabled", false);
}

@Override
Expand Down Expand Up @@ -109,7 +112,7 @@ public RestChannelConsumer prepareRequest(final RestRequest request, final NodeC
// this might be set by old clients
request.param("min_compatible_shard_node");

if (settings != null && settings.getAsBoolean("serverless.cross_project.enabled", false)) {
if (inCpsContext) {
// accept but drop project_routing param until fully supported
request.param("project_routing");
}
Expand All @@ -128,7 +131,16 @@ public RestChannelConsumer prepareRequest(final RestRequest request, final NodeC
*/
IntConsumer setSize = size -> searchRequest.source().size(size);
request.withContentOrSourceParamParserOrNull(
parser -> parseSearchRequest(searchRequest, request, parser, clusterSupportsFeature, setSize, searchUsageHolder)
parser -> parseSearchRequest(
searchRequest,
request,
parser,
clusterSupportsFeature,
setSize,
searchUsageHolder,
// This endpoint is CPS-enabled so propagate the right value.
Optional.of(inCpsContext)
)
);

return channel -> {
Expand All @@ -146,15 +158,23 @@ public RestChannelConsumer prepareRequest(final RestRequest request, final NodeC
* parameter
* @param clusterSupportsFeature used to check if certain features are available in this cluster
* @param setSize how the size url parameter is handled. {@code udpate_by_query} and regular search differ here.
* @param inCpsContext specifies if we're in CPS context.
* <br>
* true - the endpoint that's invoking this method is CPS-enabled and in a CPS/Serverless context.
* <br>
* false - the endpoint that's invoking this method is CPS-enabled but not in a CPS/Serverless context.
* <br>
* Optional.empty - the endpoint is not CPS-enabled irrespective of the environment.
*/
public static void parseSearchRequest(
SearchRequest searchRequest,
RestRequest request,
XContentParser requestContentParser,
Predicate<NodeFeature> clusterSupportsFeature,
IntConsumer setSize
IntConsumer setSize,
Optional<Boolean> inCpsContext
) throws IOException {
parseSearchRequest(searchRequest, request, requestContentParser, clusterSupportsFeature, setSize, null);
parseSearchRequest(searchRequest, request, requestContentParser, clusterSupportsFeature, setSize, null, inCpsContext);
}

/**
Expand All @@ -167,14 +187,22 @@ public static void parseSearchRequest(
* @param clusterSupportsFeature used to check if certain features are available in this cluster
* @param setSize how the size url parameter is handled. {@code udpate_by_query} and regular search differ here.
* @param searchUsageHolder the holder of search usage stats
* @param inCpsContext specifies if we're in CPS context.
* <br>
* true - the endpoint that's invoking this method is CPS-enabled and in a CPS/Serverless context.
* <br>
* false - the endpoint that's invoking this method is CPS-enabled but not in a CPS/Serverless context.
* <br>
* Optional.empty - the endpoint is not CPS-enabled irrespective of the environment.
*/
public static void parseSearchRequest(
SearchRequest searchRequest,
RestRequest request,
@Nullable XContentParser requestContentParser,
Predicate<NodeFeature> clusterSupportsFeature,
IntConsumer setSize,
@Nullable SearchUsageHolder searchUsageHolder
@Nullable SearchUsageHolder searchUsageHolder,
Optional<Boolean> inCpsContext
) throws IOException {
if (searchRequest.source() == null) {
searchRequest.source(new SearchSourceBuilder());
Expand Down Expand Up @@ -229,9 +257,17 @@ public static void parseSearchRequest(
if (searchRequest.pointInTimeBuilder() != null) {
preparePointInTime(searchRequest, request);
} else {
searchRequest.setCcsMinimizeRoundtrips(
request.paramAsBoolean("ccs_minimize_roundtrips", searchRequest.isCcsMinimizeRoundtrips())
);
if (inCpsContext.orElse(false)) {
// We're in CPS environment. MRT should not be settable by the user.
if (request.hasParam("ccs_minimize_roundtrips")) {
throw new IllegalArgumentException("Setting ccs_minimize_roundtrips is not supported in cross-project search context");
}
} else {
// Either we're in non-CPS environment or the endpoint isn't CPS enabled, so parse what's in the request.
searchRequest.setCcsMinimizeRoundtrips(
request.paramAsBoolean("ccs_minimize_roundtrips", searchRequest.isCcsMinimizeRoundtrips())
);
}
}
if (request.paramAsBoolean("force_synthetic_source", false)) {
searchRequest.setForceSyntheticSource(true);
Expand Down
17 changes: 15 additions & 2 deletions ...sync-search/src/main/java/org/elasticsearch/xpack/search/RestSubmitAsyncSearchAction.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import java.io.IOException;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.function.IntConsumer;
import java.util.function.Predicate;
Expand All @@ -39,6 +40,7 @@ public final class RestSubmitAsyncSearchAction extends BaseRestHandler {
private final SearchUsageHolder searchUsageHolder;
private final Predicate<NodeFeature> clusterSupportsFeature;
private final Settings settings;
private final boolean inCpsContext;

public RestSubmitAsyncSearchAction(SearchUsageHolder searchUsageHolder, Predicate<NodeFeature> clusterSupportsFeature) {
this(searchUsageHolder, clusterSupportsFeature, null);
Expand All @@ -52,6 +54,7 @@ public RestSubmitAsyncSearchAction(
this.searchUsageHolder = searchUsageHolder;
this.clusterSupportsFeature = clusterSupportsFeature;
this.settings = settings;
this.inCpsContext = settings != null && settings.getAsBoolean("serverless.cross_project.enabled", false);
}

@Override
Expand All @@ -71,9 +74,10 @@ protected RestChannelConsumer prepareRequest(RestRequest request, NodeClient cli
}
SubmitAsyncSearchRequest submit = new SubmitAsyncSearchRequest();

if (settings != null && settings.getAsBoolean("serverless.cross_project.enabled", false)) {
if (inCpsContext) {
// accept but drop project_routing param until fully supported
request.param("project_routing");
submit.getSearchRequest().setCcsMinimizeRoundtrips(true);
}

IntConsumer setSize = size -> submit.getSearchRequest().source().size(size);
Expand All @@ -82,7 +86,16 @@ protected RestChannelConsumer prepareRequest(RestRequest request, NodeClient cli
// them as supported. We rely on SubmitAsyncSearchRequest#validate to fail in case they are set.
// Note that ccs_minimize_roundtrips is also set this way, which is a supported option.
request.withContentOrSourceParamParserOrNull(
parser -> parseSearchRequest(submit.getSearchRequest(), request, parser, clusterSupportsFeature, setSize, searchUsageHolder)
parser -> parseSearchRequest(
submit.getSearchRequest(),
request,
parser,
clusterSupportsFeature,
setSize,
searchUsageHolder,
// This endpoint is CPS-enabled so propagate the right value.
Optional.of(inCpsContext)
)
);

if (request.hasParam("wait_for_completion_timeout")) {
Expand Down
12 changes: 11 additions & 1 deletion .../plugin/fleet/src/main/java/org/elasticsearch/xpack/fleet/rest/RestFleetSearchAction.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.function.IntConsumer;
import java.util.function.Predicate;
Expand Down Expand Up @@ -70,7 +71,16 @@ protected RestChannelConsumer prepareRequest(RestRequest request, NodeClient cli

IntConsumer setSize = size -> searchRequest.source().size(size);
request.withContentOrSourceParamParserOrNull(parser -> {
RestSearchAction.parseSearchRequest(searchRequest, request, parser, clusterSupportsFeature, setSize, searchUsageHolder);
RestSearchAction.parseSearchRequest(
searchRequest,
request,
parser,
clusterSupportsFeature,
setSize,
searchUsageHolder,
// This endpoint is not CPS-enabled.
Optional.empty()
);
String[] stringWaitForCheckpoints = request.paramAsStringArray("wait_for_checkpoints", Strings.EMPTY_ARRAY);
final long[] waitForCheckpoints = new long[stringWaitForCheckpoints.length];
for (int i = 0; i < stringWaitForCheckpoints.length; ++i) {
Expand Down
5 changes: 4 additions & 1 deletion ...ugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/rest/RestRollupSearchAction.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

import java.io.IOException;
import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.function.Predicate;

Expand Down Expand Up @@ -52,7 +53,9 @@ protected RestChannelConsumer prepareRequest(RestRequest restRequest, NodeClient
restRequest,
parser,
clusterSupportsFeature,
size -> searchRequest.source().size(size)
size -> searchRequest.source().size(size),
// This endpoint is not CPS-enabled.
Optional.empty()
)
);
RestSearchAction.validateSearchRequest(restRequest, searchRequest);
Expand Down