Skip to content

Conversation

julian-elastic
Copy link
Contributor

@julian-elastic julian-elastic commented Sep 3, 2025

Speeds up MultiClusterSpecIT by parallelizing calls in CsvTestDataLoader to create index, load data into index and create enrich policy. Drops the runtime of the first test from 117sec to 70sec for me locally in IntelliJ.

@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label v9.2.0 labels Sep 3, 2025
@julian-elastic julian-elastic added >test Issues or PRs that are addressing/adding tests Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :Analytics/ES|QL AKA ESQL and removed needs:triage Requires assignment of a team area label labels Sep 3, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@julian-elastic julian-elastic changed the title ESQL: Speed up MultiClusterSpecIT by parallelizing index creation ESQL: Speed up MultiClusterSpecIT start up by 40% by parallelizing index creation Sep 3, 2025
Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks Julian!

Copy link
Contributor

@luigidellaquila luigidellaquila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks @julian-elastic !

Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks Julian!

@julian-elastic julian-elastic merged commit 93d0896 into elastic:main Sep 5, 2025
32 of 33 checks passed
@julian-elastic julian-elastic deleted the fasterCsvTests_v2 branch September 5, 2025 00:24
jbaiera pushed a commit to jbaiera/elasticsearch that referenced this pull request Sep 5, 2025
…dex creation (elastic#134090)

Speeds up MultiClusterSpecIT by parallelizing calls in CsvTestDataLoader to create index, load data into index and create enrich policy. Drops the runtime of the first test from 117sec to 70sec for me locally in IntelliJ.
julian-elastic added a commit to julian-elastic/elasticsearch that referenced this pull request Sep 10, 2025
julian-elastic added a commit that referenced this pull request Sep 10, 2025
…izing index creation (#134090)" (#134511)

This reverts commit 93d0896 merged with #134090
Unfortunately, it caused a flaky test found by #134414

Closes #134414
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >test Issues or PRs that are addressing/adding tests v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants