Skip to content

ESQL: More specific index pattern in testMultipleBatchesWithLookupJoin #130006

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 26, 2025
Merged

ESQL: More specific index pattern in testMultipleBatchesWithLookupJoin #130006

Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8367062
Specific idx in testMultipleBatchesWithLookupJoin
alex-spies Jun 25, 2025
3748346
[CI] Auto commit changes from spotless
Jun 25, 2025
3d6619a
Merge branch 'main' into make-layout-failure-test-more-specific
alex-spies Jun 25, 2025
58364e4
Remove column count validation
alex-spies Jun 25, 2025
9fc2db3
Merge remote-tracking branch 'upstream/main' into make-layout-failure…
alex-spies Jun 25, 2025
f6b8f1b
Merge branch 'main' into make-layout-failure-test-more-specific
alex-spies Jun 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions ...n/esql/qa/server/src/main/java/org/elasticsearch/xpack/esql/qa/rest/RestEsqlTestCase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1014,12 +1014,14 @@ public void testMultipleBatchesWithLookupJoin() throws IOException {
// Create more than 10 indices to trigger multiple batches of data node execution.
// The sort field should be missing on some indices to reproduce NullPointerException caused by duplicated items in layout
for (int i = 1; i <= 20; i++) {
createIndex("idx" + i, randomBoolean(), "\"mappings\": {\"properties\" : {\"a\" : {\"type\" : \"keyword\"}}}");
createIndex("no_sort_field_idx" + i, randomBoolean(), "\"mappings\": {\"properties\" : {\"a\" : {\"type\" : \"keyword\"}}}");
Copy link
Contributor

@idegtiarenko idegtiarenko Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should go one step forward and rely on randomIdentifier() for the index name here?

Copy link
Contributor Author

@alex-spies alex-spies Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes clean up a little harder as I'd need to keep track of the index names to delete them later or make it so I can perform a wildcard delete. This should be fine, I don't think this was the root cause of the failure.

}
bulkLoadTestDataLookupMode(10);
// lookup join with and without sort
for (String sort : List.of("", "| sort integer")) {
var query = requestObjectBuilder().query(format(null, "from * | lookup join {} on integer {}", testIndexName(), sort));
var query = requestObjectBuilder().query(
format(null, "from {},no_sort_field_idx* | lookup join {} on integer {}", testIndexName(), testIndexName(), sort)
);
Map<String, Object> result = runEsql(query);
var columns = as(result.get("columns"), List.class);
assertEquals(22, columns.size());
Copy link
Member

@fang-xing-esql fang-xing-esql Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option is removing the column count validation, as this test needs to be updated every time a new field is added to the default index. It is weird where the extra column comes from though, I thought at the beginning of the tests, all existing indices are cleaned up, we should only have a deterministic number of columns in the resultset.

Copy link
Contributor Author

@alex-spies alex-spies Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assertion deleted in the next push!

Expand All @@ -1028,7 +1030,7 @@ public void testMultipleBatchesWithLookupJoin() throws IOException {
}
// clean up
for (int i = 1; i <= 20; i++) {
assertThat(deleteIndex("idx" + i).isAcknowledged(), is(true));
assertThat(deleteIndex("no_sort_field_idx" + i).isAcknowledged(), is(true));
}
}

Expand Down