Skip to content

Address es819 tsdb doc values format performance bug #135505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Sep 26, 2025
Merged

Address es819 tsdb doc values format performance bug #135505

merged 2 commits into from
Sep 26, 2025

Conversation

@martijnvg
Copy link
Member

@martijnvg martijnvg commented Sep 26, 2025

  • The DISIAccumulator should only be used during merging.
  • The tmp files created by DISIAccumulator and OffsetsAccumulator shouldn't use mmap based lucene directory.

Closes #135340

@martijnvg
Address es819 tsdb doc values format performance bug
dc5c1e4 
* The DISIAccumulator should only be used during merging.
* The tmp files created by DISIAccumulator and OffsetsAccumulator shouldn't use mmap based lucene directory.

Closes elastic#135340
@martijnvg martijnvg requested a review from dnhatn September 26, 2025 07:05
@elasticsearchmachine
Copy link
Collaborator

elasticsearchmachine commented Sep 26, 2025

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@elasticsearchmachine
Copy link
Collaborator

elasticsearchmachine commented Sep 26, 2025

Hi @martijnvg, I've created a changelog YAML for you.

dnhatn
dnhatn approved these changes Sep 26, 2025
View reviewed changes
Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Martijn!

@martijnvg martijnvg added the auto-backport Automatically create backport pull requests when merged label Sep 26, 2025
@martijnvg martijnvg merged commit 1aed2bc into elastic:main Sep 26, 2025
34 checks passed
@elasticsearchmachine
Copy link
Collaborator

elasticsearchmachine commented Sep 26, 2025

💔 Backport failed

Status Branch Result
❌ 8.19 Commit could not be cherrypicked due to conflicts
❌ 9.1 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 135505

szybia added a commit to szybia/elasticsearch that referenced this pull request Sep 26, 2025
@szybia
Merge remote-tracking branch 'upstream/main' into security-stats-with…
31892e0 
…-dls

* upstream/main: (55 commits)
  Mute org.elasticsearch.upgrades.MatchOnlyTextRollingUpgradeIT testIndexing {upgradedNodes=1} elastic#135525
  Address es819 tsdb doc values format performance bug (elastic#135505)
  Remove obsolete --add-opens from JDK API extractor tool (elastic#135445)
  [CI] Fix MergeWithFailureIT (elastic#135447)
  Increase wait time in AdaptiveAllocationsScalerServiceTests (elastic#135510)
  ES|QL: Handle multi values in FUSE (elastic#135448)
  Mute org.elasticsearch.upgrades.SyntheticSourceRollingUpgradeIT testIndexing {upgradedNodes=1} elastic#135512
  Add trust configuration for cross cluster api keys (elastic#134893)
  ESQL: Fix flakiness in SessionUtilsTests (elastic#135375)
  Mute org.elasticsearch.upgrades.LogsdbIndexingRollingUpgradeIT testIndexing {upgradedNodes=1} elastic#135511
  Mute org.elasticsearch.upgrades.MatchOnlyTextRollingUpgradeIT testIndexing {upgradedNodes=2} elastic#135325
  Require all functions to provide examples (elastic#135094)
  Mute org.elasticsearch.upgrades.SyntheticSourceRollingUpgradeIT testIndexing {upgradedNodes=2} elastic#135344
  Mute org.elasticsearch.upgrades.TextRollingUpgradeIT testIndexing {upgradedNodes=1} elastic#135236
  Mute org.elasticsearch.upgrades.TextRollingUpgradeIT testIndexing {upgradedNodes=2} elastic#135238
  Mute org.elasticsearch.upgrades.LogsdbIndexingRollingUpgradeIT testIndexing {upgradedNodes=2} elastic#135327
  Mute org.elasticsearch.upgrades.MatchOnlyTextRollingUpgradeIT testIndexing {upgradedNodes=3} elastic#135324
  Mute org.elasticsearch.upgrades.StandardToLogsDbIndexModeRollingUpgradeIT testLogsIndexing {upgradedNodes=3} elastic#135315
  ESQL: Handle right hand side of Inline Stats coming optimized with LocalRelation shortcut (elastic#135011)
  Mute org.elasticsearch.upgrades.TextRollingUpgradeIT testIndexing {upgradedNodes=3} elastic#135237
  ...
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Sep 26, 2025
@martijnvg
[9.1] Address es819 tsdb doc values format performance bug
27335d6 
Backporting elastic#135505 to 9.1 branch.

* The DISIAccumulator should only be used during merging.
* The tmp files created by DISIAccumulator and OffsetsAccumulator shouldn't use mmap based lucene directory.

Closes elastic#135340
@martijnvg martijnvg mentioned this pull request Sep 26, 2025
Merged
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Sep 26, 2025
@martijnvg
[9.1] Address es819 tsdb doc values format performance bug
9736c1d 
Backporting elastic#135505 to 9.1 branch.

* The DISIAccumulator should only be used during merging.
* The tmp files created by DISIAccumulator and OffsetsAccumulator shouldn't use mmap based lucene directory.

Closes elastic#135340
@martijnvg martijnvg mentioned this pull request Sep 26, 2025
Merged
martijnvg added a commit that referenced this pull request Sep 29, 2025
@martijnvg
[9.1] Address es819 tsdb doc values format performance bug (#135535)
e837ccd 
Backporting #135505 to 9.1 branch.

* The DISIAccumulator should only be used during merging.
* The tmp files created by DISIAccumulator and OffsetsAccumulator shouldn't use mmap based lucene directory.

Closes #135340
martijnvg added a commit that referenced this pull request Sep 29, 2025
@martijnvg
[8.19] Address es819 tsdb doc values format performance bug (#135536)
a89cdb4 
Backporting #135505 to 8.19 branch.

* The DISIAccumulator should only be used during merging.
* The tmp files created by DISIAccumulator and OffsetsAccumulator shouldn't use mmap based lucene directory.

Closes #135340
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dnhatn dnhatn dnhatn approved these changes

Assignees

No one assigned

Labels

auto-backport Automatically create backport pull requests when merged >bug :StorageEngine/Codec Team:StorageEngine v8.19.5 v9.1.5 v9.2.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Possible performance regression in 9.1.x (TSDS)

3 participants

@martijnvg @elasticsearchmachine @dnhatn