Resolve indices using original index pattern and retry the entire resolution #136009
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/session/EsqlSession.java
# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/session/EsqlSession.java
# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/PreAnalyzer.java
idegtiarenko
added
>non-issue
Team:Analytics
idegtiarenko
commented
Oct 6, 2025
| syncEsqlQueryRequest().query("FROM data,*:data | LOOKUP JOIN lookup ON key | WHERE f1 == 1") | ||
| .filter(new TermQueryBuilder("f2", 2)) | ||
| ) | ||
| ); |
Contributor
Author
There was a problem hiding this comment.
This test would fail if we only retry previously resolved subset of clusters opposed to ones resolved to all clusters without the filter.
Collaborator
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
idegtiarenko
changed the title
luigidellaquila
approved these changes
Oct 7, 2025
Contributor
luigidellaquila
left a comment
luigidellaquila
left a comment
There was a problem hiding this comment.
Same as #134218, IMHO this is good
Thanks!
| } | ||
| })); | ||
| // retrying the index resolution without index filtering. | ||
| executionInfo.clusterInfo.clear(); |
smalyshev
approved these changes
Oct 8, 2025
Contributor
smalyshev
left a comment
smalyshev
left a comment
There was a problem hiding this comment.
Looks fine. I am not 100% happy with the fact that we do other resolutions twice in the case of filters, but I hope it'll go away when we fix field caps, so for now we can live with it I think.
smalyshev
reviewed
Oct 8, 2025
| VerificationException.class, | ||
| containsString("lookup index [lookup] is not available in remote cluster [remote-b]"), | ||
| () -> runQuery( | ||
| syncEsqlQueryRequest().query("FROM data,*:data | LOOKUP JOIN lookup ON key | WHERE f1 == 1") |
Contributor
There was a problem hiding this comment.
Reminder: please add the other use case without join we've discussed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Today we parse original index pattern into EsqlExecutionInfo#clusterInfo and then assemble it back createIndexExpressionFromAvailableClusters. In CPS with flat expressions resolution logic is more complex (for example the list of remotes is no longer known beforehand). It should be delegated to field caps call using original expression. This also changes how resolution is retried. It is no longer trivial to exclude clusters with failures. Additionally
flatexpression could resolve to different set of remotes without filters. As a result we need to retry resolving dependencies such as enrich and lookup joins as well as we need to ensure they are present on a wider set of remotes.Resolves: ES-12487, ES-12978