apollo_propeller: add reconstruction types and spawn task#12007
Conversation
d7e7ed5 to
3eb4134
Compare
f8d5a40 to
fc5b531
Compare
8032e47 to
efad763
Compare
guy-starkware
left a comment
There was a problem hiding this comment.
@guy-starkware reviewed 3 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @noamsp-starkware, @ShahakShama, and @sirandreww-starkware).
crates/apollo_propeller/src/message_processor.rs line 220 at r1 (raw file):
rayon::spawn(move || { let result = rebuild_message(shards, message_root, my_shard_index, data_count, coding_count)
Is there a reason rebuild_message doesn't just emit ReconstructionResult?
guy-starkware
left a comment
There was a problem hiding this comment.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on noamsp-starkware, ShahakShama, and sirandreww-starkware).
efad763 to
b5519d6
Compare
fc5b531 to
901ad8e
Compare
sirandreww-starkware
left a comment
There was a problem hiding this comment.
@sirandreww-starkware made 2 comments and resolved 1 discussion.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on guy-starkware, noamsp-starkware, and ShahakShama).
crates/apollo_propeller/src/message_processor.rs line 220 at r1 (raw file):
Previously, guy-starkware wrote…
Is there a reason
rebuild_messagedoesn't just emit ReconstructionResult?
AI answer:
reconstruct_message_from_shards lives in the sharding utility layer — it only knows about erasure coding, Merkle trees, and padding. ReconstructionSuccess (and ReconstructionResult) are processor-level concepts private to MessageProcessor, where the tuple fields get domain-specific names (my_shard, my_shard_proof) tied to the node's role. Pushing that type down into sharding.rs would couple the generic reconstruction logic to the processor's internal model. Keeping the tuple return keeps the sharding layer composable and the boundary clean.
My answer:
I don't want spinoff workers to emit events, the message processor both 1) needs to know the result of the build to update its state and 2) needs to figure out what to emit, or if to emit in the first place.
sirandreww-starkware
left a comment
There was a problem hiding this comment.
@sirandreww-starkware resolved 1 discussion.
Reviewable status: 0 of 3 files reviewed, all discussions resolved (waiting on guy-starkware, noamsp-starkware, and ShahakShama).
sirandreww-starkware
left a comment
There was a problem hiding this comment.
@sirandreww-starkware partially reviewed 3 files.
Reviewable status: 0 of 3 files reviewed, all discussions resolved (waiting on guy-starkware, noamsp-starkware, and ShahakShama).
b5519d6 to
d9e2711
Compare
901ad8e to
21443c9
Compare
guy-starkware
left a comment
There was a problem hiding this comment.
@guy-starkware reviewed 3 files and all commit messages, and made 1 comment.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on noamsp-starkware and ShahakShama).
crates/apollo_propeller/src/message_processor.rs line 220 at r1 (raw file):
Previously, sirandreww-starkware (Andrew Luka) wrote…
AI answer:
reconstruct_message_from_shards lives in the sharding utility layer — it only knows about erasure coding, Merkle trees, and padding. ReconstructionSuccess (and ReconstructionResult) are processor-level concepts private to MessageProcessor, where the tuple fields get domain-specific names (my_shard, my_shard_proof) tied to the node's role. Pushing that type down into sharding.rs would couple the generic reconstruction logic to the processor's internal model. Keeping the tuple return keeps the sharding layer composable and the boundary clean.My answer:
I don't want spinoff workers to emit events, the message processor both 1) needs to know the result of the build to update its state and 2) needs to figure out what to emit, or if to emit in the first place.
ok. I am not familiar enough with the logical boundaries, but if you think this is better I will go with it.
ShahakShama
left a comment
There was a problem hiding this comment.
@ShahakShama made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on noamsp-starkware and sirandreww-starkware).
crates/apollo_propeller/Cargo.toml line 19 at r3 (raw file):
libp2p.workspace = true prost.workspace = true rand = { workspace = true, features = ["std", "std_rng"] }
Why?
ShahakShama
left a comment
There was a problem hiding this comment.
@ShahakShama reviewed all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on noamsp-starkware and sirandreww-starkware).

Note
Low Risk
Mostly additive, currently unused reconstruction scaffolding and a minor dependency feature tweak; limited behavior change unless/when wired into the processing flow.
Overview
Adds scaffolding in
MessageProcessorfor erasure-coded message reconstruction: introducesReconstructionResult/ReconstructionSuccesstypes and aspawn_reconstruction_taskhelper that runsreconstruct_message_from_shardson the Rayon pool and returns the result via a oneshot channel.Updates dependencies to enable
rand’sstd/std_rngfeatures and adjusts imports to include reconstruction-related types (ReconstructionError,MerkleProof).Written by Cursor Bugbot for commit 21443c9. This will update automatically on new commits. Configure here.