Skip to content

Commit 88afe92

Browse files
felixbarnyfelixbarny
committed
Merge remote-tracking branch 'origin/main' into ip-parsing-optimization
2 parents 90eacee + 5e43ea5 commit 88afe92

File tree

5 files changed

+66
-36
lines changed

5 files changed

+66
-36
lines changed

‎docs/changelog/133064.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 133064
2+
summary: Mark LOOKUP JOIN as `ExecutesOn.Any` by default
3+
area: ES|QL
4+
type: bug
5+
issues: []

‎docs/reference/elasticsearch/rest-apis/index.md‎

Lines changed: 12 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -261,7 +261,7 @@ The index APIs enable you to manage individual indices, index settings, aliases,
261261
| [Shrink index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-shrink) | Shrinks an existing index into a new index with fewer primary shards. |
262262
| [Split index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-split) | Splits an existing index into a new index with more primary shards. |
263263
| [Clone index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-clone) | Clones an existing index into a new index. |
264-
| [Manage index aliases](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-alias) | Manages index aliases. |
264+
| [Check alias](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-exists-alias) | Manages index aliases. |
265265
| [Update field mappings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-mapping) | Updates index mappings. |
266266
| [Get field mappings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-mapping) | Retrieves index mappings. |
267267
| [Get index settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings) | Retrieves settings for one or more indices. |
@@ -278,10 +278,8 @@ The index APIs enable you to manage individual indices, index settings, aliases,
278278
| [Flush index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-flush) | Performs a flush operation on one or more indices. |
279279
| [Clear index cache](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-clear-cache) | Clears caches associated with one or more indices. |
280280
| [Force merge index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) | Merges index segments to reduce their number and improve performance. |
281-
| [Freeze index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-freeze) | Freezes an index, making it read-only and minimizing its resource usage. |
282-
| [Unfreeze index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-unfreeze) | Unfreezes a frozen index, making it writeable and fully functional. |
283281
| [Rollover index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover) | Rolls over an alias to a new index when conditions are met. |
284-
| [Resolve index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-resolve) | Resolves expressions to index names, aliases, and data streams. |
282+
| [Resolve index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-resolve-index) | Resolves expressions to index names, aliases, and data streams. |
285283
| [Simulate index template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-simulate-index-template) | Simulates the application of a composable index template. |
286284
| [Simulate template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-simulate-template) | Simulates the application of a legacy index template. |
287285
| [Get mapping](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-mapping) | Retrieves mapping definitions for one or more indices. |
@@ -337,9 +335,6 @@ The ingest APIs enable you to manage tasks and resources related to ingest pipel
337335
| [Delete pipeline](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-delete-pipeline) | Deletes an ingest pipeline. |
338336
| [Simulate pipeline](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-simulate) | Simulates a document through an ingest pipeline. |
339337
| [Get built-in grok patterns](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-processor-grok) | Returns a list of built-in grok patterns. |
340-
| [Get processor types](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-processor-types) | Returns a list of available processor types. |
341-
| [Put pipeline processor](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-put-processor) | Creates or updates a custom pipeline processor. |
342-
| [Delete pipeline processor](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-delete-processor) | Deletes a custom pipeline processor. |
343338

344339
### [Licensing](https://www.elastic.co/docs/api/doc/elasticsearch/v9/group/endpoint-license)
345340

@@ -422,7 +417,7 @@ The machine learning data frame analytics APIs enbale you to perform data frame
422417
| API | Description |
423418
| --- | ----------- |
424419
| [Create a data frame analytics job](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-put-data-frame-analytics) | Creates a data frame analytics job. |
425-
| [Get data frame analytics job configuration info](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-get-data-frame-analyticss) | Retrieves configuration and results for analytics jobs. |
420+
| [Get data frame analytics job configuration info](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-get-data-frame-analytics) | Retrieves configuration and results for analytics jobs. |
426421
| [Delete a data frame analytics job](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-delete-data-frame-analytics) | Deletes a data frame analytics job. |
427422
| [Start a data frame analytics job](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-start-data-frame-analytics) | Starts a data frame analytics job. |
428423
| [Stop data frame analytics jobs](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-stop-data-frame-analytics) | Stops a running data frame analytics job. |
@@ -466,9 +461,9 @@ Query rules enable you to configure per-query rules that are applied at query ti
466461

467462
| API | Description |
468463
| --- | ----------- |
469-
| [Create or update query ruleset](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-query-ruleset-put-query-ruleset) | Creates or updates a query ruleset. |
470-
| [Get query ruleset](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-query-ruleset-get-query-ruleset) | Retrieves one or more query rulesets. |
471-
| [Delete query ruleset](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-query-ruleset-delete-query-ruleset) | Deletes a query ruleset. |
464+
| [Create or update query ruleset](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-query-rules-put-ruleset) | Creates or updates a query ruleset. |
465+
| [Get query ruleset](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-query-rules-get-ruleset) | Retrieves one or more query rulesets. |
466+
| [Delete query ruleset](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-query-rules-delete-ruleset) | Deletes a query ruleset. |
472467

473468
### [Rollup](https://www.elastic.co/docs/api/doc/elasticsearch/v9/group/endpoint-rollup)
474469

@@ -491,10 +486,10 @@ Use the script support APIs to get a list of supported script contexts and langu
491486

492487
| API | Description |
493488
| --- | ----------- |
494-
| [Add or update stored script](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-script-put-script) | Adds or updates a stored script. |
495-
| [Get stored script](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-script-get-script) | Retrieves a stored script. |
496-
| [Delete stored script](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-script-delete-script) | Deletes a stored script. |
497-
| [Execute Painless script](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-script-painless-execute) | Executes a script using the Painless language. |
489+
| [Add or update stored script](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-put-script) | Adds or updates a stored script. |
490+
| [Get stored script](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-get-script) | Retrieves a stored script. |
491+
| [Delete stored script](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-delete-script) | Deletes a stored script. |
492+
| [Execute Painless script]https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-scripts-painless-execute) | Executes a script using the Painless language. |
498493
| [Get script contexts](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-get-script-context) | Returns available script execution contexts. |
499494
| [Get script languages](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-get-script-languages) | Returns available scripting languages. |
500495

@@ -509,7 +504,6 @@ The search APIs enable you to search and aggregate data stored in {{es}} indices
509504
| [Search template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-template) | Executes a search using a stored or inline template. |
510505
| [Render search template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-render-search-template) | Renders a search template with parameters. |
511506
| [Explain search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-explain) | Explains how a document scores against a query. |
512-
| [Validate query](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-validate-query) | Validates a query without executing it. |
513507
| [Get field capabilities](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-field-caps) | Returns the capabilities of fields across indices. |
514508
| [Scroll search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-scroll) | Efficiently retrieves large numbers of results (pagination). |
515509
| [Clear scroll](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-clear-scroll) | Clears search contexts for scroll requests. |
@@ -608,9 +602,9 @@ The synonyms management APIs provide a convenient way to define and manage synon
608602
| --- | ----------- |
609603
| [Get synonym set](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-synonyms-get-synonym) | Retrieves a synonym set by ID. |
610604
| [Create of update synonym set](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-synonyms-put-synonym) | Creates or updates a synonym set. |
611-
| [Delete synonym set](https://www.elastic.co/docs/api/doc/elasticsearch/endpoint/synonyms.delete_synonym) | Deletes a synonym set. |
605+
| [Delete synonym set](https://www.elastic.co/docs/api/doc/elasticsearch/endpoint/operation/operation-synonyms-delete-synonym) | Deletes a synonym set. |
612606
| [Get synonym rule](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-synonyms-get-synonym-rule) | |
613-
| [Get synonym sets](https://www.elastic.co/docs/api/doc/elasticsearch/endpoint/synonyms.get_synonyms) | Lists all synonym sets. |
607+
| [Get synonym sets](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-synonyms-get-synonyms-sets) | Lists all synonym sets. |
614608

615609
### [Task management](https://www.elastic.co/docs/api/doc/elasticsearch/v9/group/endpoint-tasks)
616610

‎muted-tests.yml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -585,9 +585,6 @@ tests:
585585
- class: org.elasticsearch.test.rest.yaml.CcsCommonYamlTestSuiteIT
586586
method: test {p0=search/160_exists_query/Test exists query on _id field}
587587
issue: https://github.com/elastic/elasticsearch/issues/133097
588-
- class: org.elasticsearch.xpack.esql.optimizer.OptimizerVerificationTests
589-
method: testRemoteEnrichAfterCoordinatorOnlyPlans
590-
issue: https://github.com/elastic/elasticsearch/issues/133015
591588
- class: org.elasticsearch.xpack.esql.analysis.AnalyzerTests
592589
method: testNoDenseVectorFailsForMagnitude
593590
issue: https://github.com/elastic/elasticsearch/issues/133013

‎x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/logical/join/Join.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -318,7 +318,7 @@ public boolean isRemote() {
318318

319319
@Override
320320
public ExecuteLocation executesOn() {
321-
return isRemote ? ExecuteLocation.REMOTE : ExecuteLocation.COORDINATOR;
321+
return isRemote ? ExecuteLocation.REMOTE : ExecuteLocation.ANY;
322322
}
323323

324324
private void checkRemoteJoin(Failures failures) {

‎x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/OptimizerVerificationTests.java‎

Lines changed: 48 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
import static org.elasticsearch.xpack.esql.analysis.AnalyzerTestUtils.defaultLookupResolution;
2828
import static org.elasticsearch.xpack.esql.analysis.AnalyzerTestUtils.loadEnrichPolicyResolution;
2929
import static org.elasticsearch.xpack.esql.analysis.AnalyzerTestUtils.loadMapping;
30+
import static org.elasticsearch.xpack.esql.plugin.EsqlPlugin.INLINESTATS_FEATURE_FLAG;
3031
import static org.hamcrest.Matchers.containsString;
3132
import static org.hamcrest.Matchers.instanceOf;
3233

@@ -94,16 +95,40 @@ public void testRemoteEnrichAfterCoordinatorOnlyPlans() {
9495

9596
String err;
9697

98+
// Remote enrich is ok after limit
9799
plan("""
98100
FROM test
101+
| LIMIT 10
99102
| EVAL language_code = languages
100103
| ENRICH _remote:languages ON language_code
101104
| STATS count(*) BY language_name
102105
""", analyzer);
103106

107+
// Remote enrich is ok after topn
108+
plan("""
109+
FROM test
110+
| EVAL language_code = languages
111+
| SORT languages
112+
| ENRICH _remote:languages ON language_code
113+
""", analyzer);
114+
plan("""
115+
FROM test
116+
| EVAL language_code = languages
117+
| SORT languages
118+
| LIMIT 2
119+
| ENRICH _remote:languages ON language_code
120+
""", analyzer);
121+
122+
// Remote enrich is ok before pipeline breakers
104123
plan("""
105124
FROM test
125+
| EVAL language_code = languages
126+
| ENRICH _remote:languages ON language_code
106127
| LIMIT 10
128+
""", analyzer);
129+
130+
plan("""
131+
FROM test
107132
| EVAL language_code = languages
108133
| ENRICH _remote:languages ON language_code
109134
| STATS count(*) BY language_name
@@ -117,24 +142,33 @@ public void testRemoteEnrichAfterCoordinatorOnlyPlans() {
117142
| LIMIT 10
118143
""", analyzer);
119144

120-
err = error("""
145+
plan("""
121146
FROM test
122147
| EVAL language_code = languages
123-
| STATS count(*) BY language_code
124148
| ENRICH _remote:languages ON language_code
149+
| SORT language_name
125150
""", analyzer);
126-
assertThat(err, containsString("4:3: ENRICH with remote policy can't be executed after [STATS count(*) BY language_code]@3:3"));
127151

128152
err = error("""
129153
FROM test
130154
| EVAL language_code = languages
131-
| INLINESTATS count(*) BY language_code
155+
| STATS count(*) BY language_code
132156
| ENRICH _remote:languages ON language_code
133157
""", analyzer);
134-
assertThat(
135-
err,
136-
containsString("4:3: ENRICH with remote policy can't be executed after [INLINESTATS count(*) BY language_code]@3:3")
137-
);
158+
assertThat(err, containsString("4:3: ENRICH with remote policy can't be executed after [STATS count(*) BY language_code]@3:3"));
159+
160+
if (INLINESTATS_FEATURE_FLAG) {
161+
err = error("""
162+
FROM test
163+
| EVAL language_code = languages
164+
| INLINESTATS count(*) BY language_code
165+
| ENRICH _remote:languages ON language_code
166+
""", analyzer);
167+
assertThat(
168+
err,
169+
containsString("4:3: ENRICH with remote policy can't be executed after [INLINESTATS count(*) BY language_code]@3:3")
170+
);
171+
}
138172

139173
err = error("""
140174
FROM test
@@ -227,6 +261,9 @@ public void testRemoteEnrichAfterCoordinatorOnlyPlans() {
227261
assertThat(err, containsString("4:3: ENRICH with remote policy can't be executed after [CHANGE_POINT salary ON languages]@2:3"));
228262
}
229263

264+
/**
265+
* The validation should not trigger for remote enrich after a lookup join. Lookup joins can be executed anywhere.
266+
*/
230267
public void testRemoteEnrichAfterLookupJoin() {
231268
EnrichResolution enrichResolution = new EnrichResolution();
232269
loadEnrichPolicyResolution(
@@ -254,32 +291,29 @@ public void testRemoteEnrichAfterLookupJoin() {
254291
| %s
255292
""", lookupCommand), analyzer);
256293

257-
String err = error(Strings.format("""
294+
plan(Strings.format("""
258295
FROM test
259296
| EVAL language_code = languages
260297
| %s
261298
| ENRICH _remote:languages ON language_code
262299
""", lookupCommand), analyzer);
263-
assertThat(err, containsString("4:3: ENRICH with remote policy can't be executed after [" + lookupCommand + "]@3:3"));
264300

265-
err = error(Strings.format("""
301+
plan(Strings.format("""
266302
FROM test
267303
| EVAL language_code = languages
268304
| %s
269305
| ENRICH _remote:languages ON language_code
270306
| %s
271307
""", lookupCommand, lookupCommand), analyzer);
272-
assertThat(err, containsString("4:3: ENRICH with remote policy can't be executed after [" + lookupCommand + "]@3:3"));
273308

274-
err = error(Strings.format("""
309+
plan(Strings.format("""
275310
FROM test
276311
| EVAL language_code = languages
277312
| %s
278313
| EVAL x = 1
279314
| MV_EXPAND language_code
280315
| ENRICH _remote:languages ON language_code
281316
""", lookupCommand), analyzer);
282-
assertThat(err, containsString("6:3: ENRICH with remote policy can't be executed after [" + lookupCommand + "]@3:3"));
283317
}
284318

285319
public void testRemoteLookupJoinWithPipelineBreaker() {

0 commit comments

Comments
 (0)