Skip to content

resolve indices for prefixed _all expressions #137330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Nov 6, 2025
Merged

resolve indices for prefixed _all expressions #137330

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
9c7e04b
resolve indices for prefixed _all expressions
richard-dennehy Oct 29, 2025
2ecdc40
add test for origin prefix _all expression
richard-dennehy Oct 29, 2025
b19ec68
[CI] Auto commit changes from spotless
Oct 29, 2025
84ea19d
use splitIndexName instead of rewriteIndexExpression
richard-dennehy Oct 30, 2025
8bbfd8f
Merge branch 'main' into remote-project-prefixed-all-expressions
richard-dennehy Oct 30, 2025
aaea6cf
use rewriteIndexExpression again
richard-dennehy Oct 31, 2025
7812375
Merge remote-tracking branch 'upstream/main' into remote-project-pref…
richard-dennehy Nov 5, 2025
ef0417d
extend shouldExcludeLocalResolution check
richard-dennehy Nov 5, 2025
13ceafc
rename
richard-dennehy Nov 5, 2025
d3470c9
Merge branch 'main' into remote-project-prefixed-all-expressions
richard-dennehy Nov 6, 2025
9730e5e
review comment
richard-dennehy Nov 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions ...urity/src/main/java/org/elasticsearch/xpack/security/authz/IndicesAndAliasesResolver.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -340,10 +340,12 @@ ResolvedIndices resolveIndicesAndAliases(
String allIndicesPatternSelector = null;
if (indicesRequest.indices() != null && indicesRequest.indices().length > 0) {
// Always parse selectors, but do so lazily so that we don't spend a lot of time splitting strings each resolution
isAllIndices = IndexNameExpressionResolver.isAllIndices(
indicesList(indicesRequest.indices()),
(expr) -> IndexNameExpressionResolver.splitSelectorExpression(expr).v1()
);
isAllIndices = IndexNameExpressionResolver.isAllIndices(indicesList(indicesRequest.indices()), (expr) -> {
var unprefixed = crossProjectModeDecider.resolvesCrossProject(replaceable)
? RemoteClusterAware.splitIndexName(expr)[1]
: expr;
return unprefixed != null ? IndexNameExpressionResolver.splitSelectorExpression(unprefixed).v1() : null;
Copy link
Member

@ywangd ywangd Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think unprefixed cannot be null?

});
if (isAllIndices) {
// This parses the single all-indices expression for a second time in this conditional branch, but this is better than
// parsing a potentially big list of indices on every request.
Expand Down Expand Up @@ -394,12 +396,13 @@ ResolvedIndices resolveIndicesAndAliases(
replaceable.getProjectRouting(),
authorizedProjects
);
remoteIndices = CrossProjectIndexExpressionsRewriter.rewriteIndexExpression(
final var rewritten = CrossProjectIndexExpressionsRewriter.rewriteIndexExpression(
indexExpression,
resolvedProjects.originProjectAlias(),
resolvedProjects.allProjectAliases()
).remoteExpressions();
if (resolvedProjects.originProject() == null) {
);
remoteIndices = rewritten.remoteExpressions();
if (resolvedProjects.originProject() == null || rewritten.localExpression() == null) {
shouldExcludeLocalResolution = true;
}
}
Expand Down
113 changes: 113 additions & 0 deletions .../src/test/java/org/elasticsearch/xpack/security/authz/IndicesAndAliasesResolverTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@

import static org.elasticsearch.action.ResolvedIndexExpression.LocalIndexResolutionResult.CONCRETE_RESOURCE_NOT_VISIBLE;
import static org.elasticsearch.action.ResolvedIndexExpression.LocalIndexResolutionResult.CONCRETE_RESOURCE_UNAUTHORIZED;
import static org.elasticsearch.action.ResolvedIndexExpression.LocalIndexResolutionResult.NONE;
import static org.elasticsearch.action.ResolvedIndexExpression.LocalIndexResolutionResult.SUCCESS;
import static org.elasticsearch.cluster.metadata.DataStreamTestHelper.newInstance;
import static org.elasticsearch.test.ActionListenerUtils.anyActionListener;
Expand Down Expand Up @@ -2956,6 +2957,118 @@ public void testCrossProjectSearchSelectorsNotAllowed() {
assertThat(exception.getMessage(), equalTo("Selectors are not currently supported but was found in the expression [_all::data]"));
}

public void testResolveAllWithWildcardRemotePrefix() {
when(crossProjectModeDecider.resolvesCrossProject(any(IndicesRequest.Replaceable.class))).thenReturn(true);

var request = new SearchRequest().indices("*:_all");
request.indicesOptions(IndicesOptions.fromOptions(randomBoolean(), randomBoolean(), true, true));
var resolvedIndices = defaultIndicesResolver.resolveIndicesAndAliases(
"indices:/" + randomAlphaOfLength(8),
request,
projectMetadata,
buildAuthorizedIndices(user, TransportSearchAction.TYPE.name()),
new TargetProjects(
createRandomProjectWithAlias("local"),
List.of(createRandomProjectWithAlias("P1"), createRandomProjectWithAlias("P2"), createRandomProjectWithAlias("P3"))
)
);

var expectedIndices = new String[] { "bar", "foobarfoo", "bar-closed", "foofoobar", "foofoo-closed", "foofoo" };

assertThat(resolvedIndices.getLocal(), contains(expectedIndices));
assertThat(resolvedIndices.getRemote(), containsInAnyOrder("P1:_all", "P2:_all", "P3:_all"));

final var resolved = request.getResolvedIndexExpressions();
assertThat(resolved, is(notNullValue()));
assertThat(
resolved.expressions(),
contains(resolvedIndexExpression("*:_all", Set.of(expectedIndices), SUCCESS, Set.of("P1:_all", "P2:_all", "P3:_all")))
);
}

public void testResolveAllWithLocalPrefix() {
when(crossProjectModeDecider.resolvesCrossProject(any(IndicesRequest.Replaceable.class))).thenReturn(true);

var expression = randomBoolean() ? "local:_all" : "_origin:_all";
var request = new SearchRequest().indices(expression);
request.indicesOptions(IndicesOptions.fromOptions(randomBoolean(), randomBoolean(), true, true));
var resolvedIndices = defaultIndicesResolver.resolveIndicesAndAliases(
"indices:/" + randomAlphaOfLength(8),
request,
projectMetadata,
buildAuthorizedIndices(user, TransportSearchAction.TYPE.name()),
new TargetProjects(
createRandomProjectWithAlias("local"),
List.of(createRandomProjectWithAlias("P1"), createRandomProjectWithAlias("P2"), createRandomProjectWithAlias("P3"))
)
);

var expectedIndices = new String[] { "bar", "foobarfoo", "bar-closed", "foofoobar", "foofoo-closed", "foofoo" };

assertThat(resolvedIndices.getLocal(), contains(expectedIndices));
assertThat(resolvedIndices.getRemote(), is(empty()));

final var resolved = request.getResolvedIndexExpressions();
assertThat(resolved, is(notNullValue()));
assertThat(resolved.expressions(), contains(resolvedIndexExpression(expression, Set.of(expectedIndices), SUCCESS, Set.of())));
}

public void testResolveAllWithRemotePrefix() {
when(crossProjectModeDecider.resolvesCrossProject(any(IndicesRequest.Replaceable.class))).thenReturn(true);

var request = new SearchRequest().indices("P*:_all");
request.indicesOptions(IndicesOptions.fromOptions(randomBoolean(), randomBoolean(), true, true));
var resolvedIndices = defaultIndicesResolver.resolveIndicesAndAliases(
"indices:/" + randomAlphaOfLength(8),
request,
projectMetadata,
buildAuthorizedIndices(user, TransportSearchAction.TYPE.name()),
new TargetProjects(
createRandomProjectWithAlias("local"),
List.of(createRandomProjectWithAlias("P1"), createRandomProjectWithAlias("P2"), createRandomProjectWithAlias("P3"))
)
);

assertThat(resolvedIndices.getLocal(), is(empty()));
assertThat(resolvedIndices.getRemote(), contains("P1:_all", "P2:_all", "P3:_all"));

final var resolved = request.getResolvedIndexExpressions();
assertThat(resolved, is(notNullValue()));
assertThat(
resolved.expressions(),
contains(resolvedIndexExpression("P*:_all", Set.of(), NONE, Set.of("P1:_all", "P2:_all", "P3:_all")))
);
}

public void testResolveIndexWithRemotePrefix() {
Copy link
Contributor Author

@richard-dennehy richard-dennehy Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that I could just set all cross project requests to go down the "all indices" path and all the tests would pass, even though this is obviously wrong; I've added this test which fails under that scenario

Copy link
Member

@ywangd ywangd Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all the tests would pass

Do you mean all tests in this test class? I am surprised unless all existing tests are effectively resolving to all accessible indices. Is that the case? Also, I assume tests in other places, e.g. the serverless REST test, should fail?

Copy link
Contributor Author

@richard-dennehy richard-dennehy Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's because the majority of the other tests in this file have CPS mode disabled; I'll see if the REST test guards against this, but I don't want it to be the only test that would fail - it makes the feedback loop too long

Copy link
Contributor Author

@richard-dennehy richard-dennehy Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed that the REST test fails when this functionality is implemented incorrectly

when(crossProjectModeDecider.resolvesCrossProject(any(IndicesRequest.Replaceable.class))).thenReturn(true);

var request = new SearchRequest().indices("*:bar");
request.indicesOptions(IndicesOptions.fromOptions(randomBoolean(), randomBoolean(), true, true));
var resolvedIndices = defaultIndicesResolver.resolveIndicesAndAliases(
"indices:/" + randomAlphaOfLength(8),
request,
projectMetadata,
buildAuthorizedIndices(user, TransportSearchAction.TYPE.name()),
new TargetProjects(
createRandomProjectWithAlias("local"),
List.of(createRandomProjectWithAlias("P1"), createRandomProjectWithAlias("P2"), createRandomProjectWithAlias("P3"))
)
);

var expectedIndices = new String[] { "bar" };

assertThat(resolvedIndices.getLocal(), contains(expectedIndices));
assertThat(resolvedIndices.getRemote(), containsInAnyOrder("P1:bar", "P2:bar", "P3:bar"));

final var resolved = request.getResolvedIndexExpressions();
assertThat(resolved, is(notNullValue()));
assertThat(
resolved.expressions(),
contains(resolvedIndexExpression("*:bar", Set.of(expectedIndices), SUCCESS, Set.of("P1:bar", "P2:bar", "P3:bar")))
);
}

private void assertIndicesMatch(IndicesRequest.Replaceable request, String expression, List<String> indices, String[] expectedIndices) {
assertThat(indices, hasSize(expectedIndices.length));
assertThat(request.indices().length, equalTo(expectedIndices.length));
Expand Down