starknet_patricia_storage: spawn blocking tasks for rocksDB reads#12168
starknet_patricia_storage: spawn blocking tasks for rocksDB reads#12168nimrod-starkware wants to merge 1 commit intographite-base/12168from
Conversation
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
d0eb009 to
f0d1e7b
Compare
25d8851 to
c241a58
Compare
f0d1e7b to
a92bc6a
Compare
c241a58 to
94e1db2
Compare
a92bc6a to
115e961
Compare
94e1db2 to
a72e9b6
Compare
115e961 to
81de56c
Compare
a72e9b6 to
89917c9
Compare
81de56c to
1e672eb
Compare
1744c70 to
5c29a32
Compare
558c573 to
33d4bdf
Compare
5c29a32 to
0b1a9e6
Compare
| starknet-types-core.workspace = true | ||
| starknet_api.workspace = true | ||
| thiserror.workspace = true | ||
| tokio.workspace = true |
There was a problem hiding this comment.
Redundant tokio dev-dependency after promoting to dependency
Low Severity
tokio.workspace = true now appears in both [dependencies] (line 34, newly added) and [dev-dependencies] (line 41, pre-existing). Since both use workspace = true with no additional features, the dev-dependency entry is now redundant — regular dependencies are automatically available in tests and benchmarks.
Additional Locations (1)
0b1a9e6 to
dda84d4
Compare
33d4bdf to
a600409
Compare
a600409 to
d9b393f
Compare
dda84d4 to
8c2f0f1
Compare
ArielElp
left a comment
There was a problem hiding this comment.
@ArielElp reviewed 4 files and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @nimrod-starkware).
crates/starknet_patricia_storage/src/rocksdb_storage.rs line 297 at r1 (raw file):
let keys: Vec<Vec<u8>> = keys.iter().map(|k| k.0.clone()).collect(); spawn_blocking(move || { let raw_keys = keys.iter().map(|k| k.as_slice());
why is this needed? can't multi_get just get keys?
crates/starknet_patricia_storage/src/rocksdb_storage.rs line 303 at r1 (raw file):
.collect::<Result<Vec<_>, PatriciaStorageError>>() }) .await?
why not use the old structure? hopefuly this avoids map_err and type annotations for the compiler in collect
spawn_blocking(move || {
let res = db
.multi_get(keys)
.into_iter()
.map(|r| r.map(|opt| opt.map(DbValue)))
.collect::<Result<_, _>>()?;
Ok(res)
})
.await?
ArielElp
left a comment
There was a problem hiding this comment.
@ArielElp made 1 comment.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @nimrod-starkware).
crates/starknet_patricia_storage/src/storage_trait.rs line 41 at r1 (raw file):
#[error(transparent)] Deserialization(#[from] DeserializationError), #[error(transparent)]
Why the new error, possible from awaiting spawn_blocking?
8c2f0f1 to
f55e97b
Compare
d9b393f to
a39458d
Compare
nimrod-starkware
left a comment
There was a problem hiding this comment.
@nimrod-starkware made 2 comments.
Reviewable status: 2 of 4 files reviewed, 4 unresolved discussions (waiting on ArielElp).
crates/starknet_patricia_storage/src/rocksdb_storage.rs line 297 at r1 (raw file):
Previously, ArielElp wrote…
why is this needed? can't multi_get just get keys?
When you call spawn_blocking, you can't pass references unless they have 'static lifetime. (this is why i need to clone)
crates/starknet_patricia_storage/src/storage_trait.rs line 41 at r1 (raw file):
Previously, ArielElp wrote…
Why the new error, possible from awaiting
spawn_blocking?
exactly
f55e97b to
be9859b
Compare
a39458d to
9fd588f
Compare
nimrod-starkware
left a comment
There was a problem hiding this comment.
@nimrod-starkware made 1 comment.
Reviewable status: 2 of 4 files reviewed, 4 unresolved discussions (waiting on ArielElp).
crates/starknet_patricia_storage/src/rocksdb_storage.rs line 303 at r1 (raw file):
Previously, ArielElp wrote…
why not use the old structure? hopefuly this avoids
map_errand type annotations for the compiler incollectspawn_blocking(move || { let res = db .multi_get(keys) .into_iter() .map(|r| r.map(|opt| opt.map(DbValue))) .collect::<Result<_, _>>()?; Ok(res) }) .await?
This is because the new error variant that can happen as result of joining
ArielElp
left a comment
There was a problem hiding this comment.
@ArielElp reviewed 2 files and all commit messages, made 1 comment, and resolved 2 discussions.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on nimrod-starkware).
crates/starknet_patricia_storage/src/rocksdb_storage.rs line 303 at r1 (raw file):
Previously, nimrod-starkware wrote…
This is because the new error variant that can happen as result of joining
But PatriciaStorageError has two transparent variants, one for Rocks errors and one for join errors, so shouldn't ? work?



Note
Medium Risk
Changes the execution model and error surface of RocksDB reads; while localized, it can impact performance and failure modes (e.g., task join/cancellation errors) under load.
Overview
RocksDB read operations in
RocksDbStorage(get/mget) are now executed viatokio::task::spawn_blockingto avoid blocking async executors, with results/errors propagated back through the asyncStorageAPI.To support this, the crate adds a
tokiodependency and extendsPatriciaStorageErrorwith aJoinErrorvariant;DeserializationError::ValueErroris tightened to requireSendso it can cross task boundaries.Written by Cursor Bugbot for commit 9fd588f. This will update automatically on new commits. Configure here.