Make RocksDB config dependant of current hardware #3740
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
This was referenced Apr 8, 2025
ndr-ds
force-pushed
the
04-08-make_rocksdb_config_dependant_of_current_hardware
branch
from
205506b to
b3ed57c
Compare
ndr-ds
force-pushed
the
03-27-all_tasks_wait_before_starting
branch
from
d9ef4ef to
538e084
Compare
Comment on lines
320
to
328
| options.set_level_zero_slowdown_writes_trigger(12); | ||
| options.set_level_zero_stop_writes_trigger(20); | ||
| options.increase_parallelism((num_cpus / 2).max(1)); | ||
| options.set_max_background_jobs((num_cpus / 3).max(1)); |
Contributor
There was a problem hiding this comment.
I think it would be useful to comment on the reason for choosing such parameters.
About commenting vs not commenting a good heuristic is to comment on things that
would take 10 minutes for a developer to find out. Here, I do not see why you chose
num_cpus / 2 and num_cpus / 3. That feels arbitrary and so deserves comment.
ndr-ds
force-pushed
the
04-08-make_rocksdb_config_dependant_of_current_hardware
branch
from
b3ed57c to
cd7e8d6
Compare
ndr-ds
force-pushed
the
03-27-all_tasks_wait_before_starting
branch
from
538e084 to
a8c7917
Compare
ndr-ds
force-pushed
the
04-08-make_rocksdb_config_dependant_of_current_hardware
branch
from
cd7e8d6 to
212506f
Compare
ndr-ds
changed the base branch from
03-27-all_tasks_wait_before_starting
to
graphite-base/3740
ndr-ds
force-pushed
the
graphite-base/3740
branch
from
344a842 to
4029c25
Compare
ndr-ds
force-pushed
the
04-08-make_rocksdb_config_dependant_of_current_hardware
branch
from
212506f to
936230c
Compare
ndr-ds
changed the base branch from
graphite-base/3740
to
03-27-all_tasks_wait_before_starting
ndr-ds
force-pushed
the
03-27-all_tasks_wait_before_starting
branch
from
4029c25 to
87a1499
Compare
ndr-ds
force-pushed
the
04-08-make_rocksdb_config_dependant_of_current_hardware
branch
from
936230c to
0840bd6
Compare
This was referenced Apr 11, 2025
Closed
Merged
Contributor
Author
ndr-ds
changed the base branch from
03-27-all_tasks_wait_before_starting
to
graphite-base/3740
ndr-ds
force-pushed
the
04-08-make_rocksdb_config_dependant_of_current_hardware
branch
from
0840bd6 to
5ca761a
Compare
ndr-ds
deleted the
04-08-make_rocksdb_config_dependant_of_current_hardware
branch
This was referenced Apr 14, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Previously the RocksDB settings were not dependant on the current hardware
Proposal
Make them dependent on the current hardware. These settings seem to work well even for the benchmarks, for example.
Test Plan
Ran benchmarks with these settings
Release Plan