Skip to content

[9.0] Speed up stored field spec management (#122645) #123915

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 3, 2025
Merged

[9.0] Speed up stored field spec management (#122645) #123915

merged 1 commit into from
Mar 3, 2025

Conversation

@nik9000
Copy link
Member

@nik9000 nik9000 commented Mar 3, 2025

Backports the following commits to 9.0:

@nik9000
Speed up stored field spec management (elastic#122645)
148a15f 
When you load stored fields from lucene you have get all of your ducks
in a row first or it'll be really slow. You need to get a list of all of
the fields you want so you can ignore the ones that you don't. We do
this with the `StoredFieldsSpec` which is immutable and has a `merge`
method. It's quite common to merge a few dozen of these specs together
to prepare for the fetch phase or for a new segment when loading fields
from ESQL.

When I was loading thousands of fields in ESQL I noticed that the merge
was slowing things down marginally. This skips a big chunk of the merge
in the common case that we don't have to load any named stored fields.
@nik9000 nik9000 added :Search Foundations/Search Catch all for Search Foundations >non-issue auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch labels Mar 3, 2025
@elasticsearchmachine elasticsearchmachine mentioned this pull request Mar 3, 2025
Merged
@elasticsearchmachine elasticsearchmachine merged commit 685f60b into elastic:9.0 Mar 3, 2025
16 checks passed
@nik9000 nik9000 deleted the backport/9.0/pr-122645 branch March 3, 2025 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport >non-issue :Search Foundations/Search Catch all for Search Foundations Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch v9.0.1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nik9000 @elasticsearchmachine