This repository was archived by the owner on Oct 30, 2025. It is now read-only.
groot/internal/rcompress: reuse buffers when possible#1023
Draft
bburghgr wants to merge 2 commits intogo-hep:mainfrom
Draft
groot/internal/rcompress: reuse buffers when possible#1023bburghgr wants to merge 2 commits intogo-hep:mainfrom
bburghgr wants to merge 2 commits intogo-hep:mainfrom
Conversation
bburghgr
force-pushed
the
rcompresspool
branch
from
8eb1422 to
eb21700
Compare
Member
|
yeah, the results are a bit of a let down. perhaps we could revisit this when/if a generic |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #712, but the current implementation is a bit delicate (from pre-generics
syncuse), so I'm not entirely sure it's worth it for the marginal performance improvements.This reuses buffers for
lz4,zlibandzstd. The other compression algs did not appear to expose an API allowing for buffer reuse. All of this should be threadsafe, but hit hasn't been tested very thoroughly. Nothing is flagged if I build groot-bench with-raceand generate files or run benchmarks, for whatever that's worth.Any performance gains would come from reducing allocations, seen in e.g.
go tool pprof -alloc-spaceon a memprofile, along with a reduction in gc frequency from the reduced allocation rate. There's very little change to overall throughput or memory use (small improvements in some cases):