Skip to content

Conversation

@joegallo
Copy link
Contributor

@joegallo joegallo commented Mar 27, 2025

Follow up to #125527

I figured I'd run some heap dumps to put actual numbers on my 10x claim from #125527. Anyway, this version really does decrease the size of the objects in memory by almost precisely 10x.

Before:
Screenshot 2025-03-27 at 3 02 05 PM

After:
Screenshot 2025-03-27 at 3 05 32 PM

@joegallo joegallo added :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >refactoring Team:Data Management Meta label for data/management team auto-backport Automatically create backport pull requests when merged v8.19.0 v9.1.0 labels Mar 27, 2025
@joegallo joegallo requested a review from masseyke March 27, 2025 18:42
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@joegallo
Copy link
Contributor Author

Actually, though, for those following along at home, while this is 10x better, it's still absurdly wasteful, since we're not doing any deduplication of low cardinality Strings:

Screenshot 2025-03-27 at 3 20 01 PM

I'll add a follow up PR where we'll do even better.

Copy link
Member

@masseyke masseyke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@joegallo joegallo merged commit 37f6ebe into elastic:main Mar 27, 2025
17 checks passed
@joegallo joegallo deleted the cache-record-for-maxmind-city branch March 27, 2025 20:07
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.x

@joegallo
Copy link
Contributor Author

Related to #96116.

joegallo added a commit to joegallo/elasticsearch that referenced this pull request Mar 28, 2025
omricohenn pushed a commit to omricohenn/elasticsearch that referenced this pull request Mar 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >refactoring Team:Data Management Meta label for data/management team v8.19.0 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants