Decrease block cache size, increase execution state cache size #4544
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Sep 11, 2025
Merged
Contributor
Author
afck
approved these changes
Sep 11, 2025
| value_cache::ValueCache, | ||
| }; | ||
|
|
||
| const BLOCK_CACHE_SIZE: usize = 5_000; |
Contributor
There was a problem hiding this comment.
If it really OOMed with 10_000 I'd do more than just halve it. I'd go down to 1_000 or even 100 maybe?
Contributor
Author
There was a problem hiding this comment.
In principle I agree, but the more we lower this the more we seem to be penalized in performance... So 5k seems like a good compromise
| use super::ValueCache; | ||
|
|
||
| /// Test cache size for unit tests. | ||
| const TEST_CACHE_SIZE: usize = 1000; |
Contributor
There was a problem hiding this comment.
Why not 10? Does it really make a difference in any of the tests? Otherwise it will just make the tests slower.
Contributor
Author
There was a problem hiding this comment.
True, 10 should be fine. Keep in mind that the slowness we would get from this is from the initial memory allocation only, because if it's some long running test it would make the test faster, actually, as more things would be cached.
For these tests that trade-off doesn't seem to make sense though, so 10 indeed makes more sense.
Contributor
|
We already have size-based cache in the repository: BlockCache in the Maybe as a follow-up though. |
ndr-ds
force-pushed
the
09-09-decrease_block_cache_size_increase_execution_state_cache_size
branch
from
73a8b47 to
d7b6485
Compare
ndr-ds
force-pushed
the
09-03-continuous_memory_profiling
branch
from
38ab4bf to
7a2e2e9
Compare
This was referenced Sep 12, 2025
Draft
ndr-ds
changed the base branch from
09-03-continuous_memory_profiling
to
graphite-base/4544
ndr-ds
force-pushed
the
09-09-decrease_block_cache_size_increase_execution_state_cache_size
branch
from
d7b6485 to
62be5ef
Compare
ndr-ds
force-pushed
the
graphite-base/4544
branch
from
7a2e2e9 to
15d2858
Compare
ndr-ds
changed the base branch from
graphite-base/4544
to
09-15-add_memory_profiling_dashboard
This was referenced Sep 15, 2025
ndr-ds
force-pushed
the
09-09-decrease_block_cache_size_increase_execution_state_cache_size
branch
from
62be5ef to
f768ed5
Compare
ndr-ds
force-pushed
the
09-15-add_memory_profiling_dashboard
branch
from
15d2858 to
987f950
Compare
ndr-ds
force-pushed
the
09-09-decrease_block_cache_size_increase_execution_state_cache_size
branch
from
f768ed5 to
9f0c9c4
Compare
ndr-ds
changed the base branch from
09-15-add_memory_profiling_dashboard
to
graphite-base/4544
ndr-ds
force-pushed
the
graphite-base/4544
branch
from
0dd0d2e to
d61401d
Compare
ndr-ds
force-pushed
the
09-09-decrease_block_cache_size_increase_execution_state_cache_size
branch
from
9f0c9c4 to
5bd75ef
Compare
ndr-ds
changed the base branch from
graphite-base/4544
to
09-15-add_memory_profiling_dashboard
ndr-ds
force-pushed
the
09-09-decrease_block_cache_size_increase_execution_state_cache_size
branch
from
5bd75ef to
fe923dd
Compare
ndr-ds
force-pushed
the
09-15-add_memory_profiling_dashboard
branch
from
d61401d to
9e25a21
Compare
ndr-ds
force-pushed
the
09-09-decrease_block_cache_size_increase_execution_state_cache_size
branch
from
fe923dd to
8be9c4e
Compare
ndr-ds
force-pushed
the
09-15-add_memory_profiling_dashboard
branch
from
9e25a21 to
3ab6fa5
Compare
ndr-ds
force-pushed
the
09-09-decrease_block_cache_size_increase_execution_state_cache_size
branch
from
8be9c4e to
91a76d1
Compare
ndr-ds
force-pushed
the
09-15-add_memory_profiling_dashboard
branch
from
3ab6fa5 to
513abd9
Compare
graphite-app
bot
changed the base branch from
09-15-add_memory_profiling_dashboard
to
graphite-base/4544
ndr-ds
force-pushed
the
09-09-decrease_block_cache_size_increase_execution_state_cache_size
branch
from
91a76d1 to
56ecc30
Compare
ndr-ds
force-pushed
the
graphite-base/4544
branch
from
513abd9 to
8916c0b
Compare
Merge activity
|
ndr-ds
deleted the
09-09-decrease_block_cache_size_increase_execution_state_cache_size
branch
ma2bd
pushed a commit
that referenced
this pull request
Sep 21, 2025
## Motivation After a while, while benchmarking, some of our shards start OOMing. After inspecting memory profiles, it seems that the culprit is the `ValueCache` for blocks specifically. ## Proposal Lower the size of the cache so that it fills up and start evicting entries before making the VM run out of memory. ## Test Plan Deploy a network with this code, benchmark it for a while and make sure that memory goes flat ## Release Plan - Nothing to do / These changes follow the usual release cycle.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
After a while, while benchmarking, some of our shards start OOMing. After inspecting memory profiles, it seems that the culprit is the
ValueCachefor blocks specifically.Proposal
Lower the size of the cache so that it fills up and start evicting entries before making the VM run out of memory.
Test Plan
Deploy a network with this code, benchmark it for a while and make sure that memory goes flat
Release Plan