feat: enforce "one `BlockComponent` per `CompletedRange`" + replay by ksn6 · Pull Request #575 · anza-xyz/alpenglow

ksn6 · 2025-10-30T06:21:29Z

Problem and Summary of Changes

We:

Enforce a bijection between BlockComponents and CompletedRanges
Introduce BlockComponentProcessor, allowing us to parse BlockComponents and enforce consensus rules in replay

local-cluster/tests/local_cluster.rs

core/src/block_creation_loop.rs

AshwinSekar · 2025-11-04T20:42:03Z

ledger/src/blockstore_processor.rs

+            if bank.parent_slot() >= first_alpenglow_slot {
+                let mut verifier = bank.block_component_verifier.write().unwrap();
+                for marker in slot_components.iter().filter_map(|bc| bc.as_marker()) {
+                    verifier.on_marker(


How will this work for UpdateParent? I'm thinking we should keep the ordering relative to the entries when processing

UpdateParent won't have any processing in block_component_verifier.rs; all UpdateParent-related checks will happen on shred ingest.

I'm thinking we should keep the ordering relative to the entries when processing

not sure I follow

ah right, that includes killing this bank if needed right?

And for the second point, maybe it doesn't matter for the current BlockMarkers e.g. header / footer, but if we had new block markers that changed some sort of bank state it seems the state change would be non-deterministic based on how the completed ranges filled in?

ah right, that includes killing this bank if needed right?

correct; it's unclear as to whether we'll want to have the direct invocation to kill the bank in blockstore, although blockstore would certainly initiate some sort of signal to kill the bank

And for the second point, maybe it doesn't matter for the current BlockMarkers e.g. header / footer, but if we had new block markers that changed some sort of bank state it seems the state change would be non-deterministic based on how the completed ranges filled in?

it's case-dependent, though we could abstractly consider:

ensuring that BlockComponentVerifier's state is identical irrespective of the order in which block components are added (this property is currently true)

placing bank-modifying logic in the finish() function, which purely operates on BlockComponentVerifier's state and is currently invoked right before the bank is frozen

this is unless we don't want to just process the new bank-modifying block component on shred ingest

My suggestion would be to process the block marker inline, also ensures that we don't have to wait until the end to throw away a block if some marker is invalid

not sure what you mean by "process the block marker inline" - do you mean "online?"

if so, BlockComponentVerifier accomplishes this via on_marker(), which is invoked in blockstore_processor.rs; i.e., at the first sight of a mistake (e.g., a double header / footer), we error out

Yeah sorry online.
Basically what i'm saying is assume you have FEC sets that look like this:

BlockMarker1 -> EntryBatch1 -> BlockMarker2 -> EntryBatch2

It seems we could have all sort of execution depending on the completed data ranges at the time.
I would expect to process these in the order that they appear in the block but:

If I receive these FEC sets one at a time this is true

If I receive all 4 FEC sets, the code written would process BM1 -> BM2 -> EB1 -> EB2

Again maybe this is not an issue because for the current BlockMarkers order doesn't matter, but I think it's a brittle assumption. For example I'm in the process of adding a new BlockMarker for migration that stores the Alpenglow genesis information in an onchain account - If EB1 interacts with this account then we could have divergent behavior

nice, makes sense! just fixed this

ledger/src/blockstore_processor.rs

akhi3030

i haven't wrapped my head around the entire PR but some minor feedback.

runtime/src/block_component_verifier.rs

runtime/src/block_component_processor.rs

ledger/src/blockstore_processor.rs

ksn6 · 2025-11-08T21:17:47Z

for readers, to address #575 (comment) easily, we'll be modifying solana-foundation/solana-improvement-documents#337 to enforce that one CompletedRange exactly corresponds to one BlockComponent

this is equivalent to saying that the DATA_COMPLETE_SHRED flag on shreds serves as a delimiter between BlockComponent

ksn6 · 2025-11-09T01:08:01Z

ledger/src/blockstore.rs

-                            })
-                            .map_err(|e| {
+                        let components =
+                            BlockComponent::from_bytes_multiple(&payload).map_err(|e| {


We can entirely remove from_bytes_multiple here and just go with from_bytes. This PR is getting large, though - will do this in a separate PR, as indicated in the below TODO.

wen-coding · 2025-11-09T00:05:39Z

core/src/replay_stage.rs

+                        .block_component_processor
+                        .read()
+                        .unwrap()
+                        .finish(migration_status)


nit: I don't like the name finish, it sounds like you are trying to finish migration. But you are really checking whether the block format is consistent with migration status right?

given a block, BlockComponentProcessor accomplishes two things:

returns an error, resulting in the block marked dead, if the block violates an invariant; sample invariants include # of block headers == 1, # of block footers == 1, etc. a follow-up PR will enforce the clock checks, e.g.

soon, apply state changes that certain block components opine on; e.g., update account rewards, the clock sysvar, etc.

the finish() function reasons about the sufficient statistics gathered by the BlockComponentProcessor during accumulation in confirm_slot, and helps accomplish the two above goals. for now, all we do is to enforce that both a block header and footer are present.

right now, the specific invariants we enforce depend on migration_status, which is why we need to pass migration_status in; this being said, I think the name is reasonable, since we'll eventually be throwing migration_status entirely away.

It is okay for now, but it gets tricky when we apply state changes I guess.

wen-coding · 2025-11-09T16:22:51Z

ledger/src/blockstore.rs

-                            })
+                            })?;
+
+                        // Enforce that completed ranges have precisely one BlockComponent.


nit: can you clarify how the caller can ensure this is true? And it should be specified in the doc as well.

once this PR lands, I'll be modifying SIMD-0337 to ensure that this statement is true

in a follow-up PR, I'll be replacing the block component parsing with parsing a single component rather than a Vec<BlockComponent> and ensuring that the length is 1

wen-coding · 2025-11-09T16:23:31Z

ledger/src/blockstore.rs

+                        // this assumption.
+                        if components.len() != 1 {
+                            return Err(BlockstoreError::InvalidShredData(Box::new(
+                                bincode::ErrorKind::Custom(format!(


nit: Is it worth introducing new error type rather than fitting into this one?

this code is going away in a follow-up PR; see #575 (comment)

wen-coding · 2025-11-09T16:25:55Z

ledger/src/blockstore.rs

+    ///   completed_ranges = [..., (s_i..e_i), (s_i+1..e_i+1), ...]
+    /// Then, the following statements are true:
+    ///   s_i < e_i == s_i+1 < e_i+1
+    fn get_slot_components_in_block(


nit: explain the difference from get_slot_entries_in_block maybe?

could you clarify? return type for getting slot components is Result<Vec<BlockComponent>> while the that of entries is Result<Vec<Entry>>; I think the comments elaborate a bit further on this

I mean, we now seem to almost identical comments between the two functions, would be good to mention difference, but I'm fine either way.

wen-coding · 2025-11-09T16:27:37Z

ledger/src/blockstore.rs

+    fn get_slot_entries_in_block(
+        &self,
+        slot: Slot,
+        completed_ranges: &CompletedRanges,


Why are we changing CompletedRanges to &CompletedRanges here?

get_slot_components_with_shred_info returns an owned completed_ranges, but also needs to invoke get_slot_components_in_block, so get_slot_components_in_block must take in &CompletedRanges

get_slot_entries_in_block invokes get_slot_components_in_block, so we might as well pass in a ref rather than an owned version of CompletedRanges, since an owned version here is more restrictive unnecessarily

ledger/src/blockstore_processor.rs

runtime/src/bank.rs

runtime/src/block_component_processor.rs

wen-coding · 2025-11-09T16:49:51Z

core/src/replay_stage.rs


+                // Verify and process block components (e.g., header, footer) before freezing
+                // Only verify blocks that were replayed from blockstore (not leader blocks)
+                if !is_leader_block {


Is there ever a case I should pay attention to my own block_components?

For example, a standby sending out block in my name, putting in block component I don't agree with?

no reason to do this - if a validator wishes to modify block component dissemination, they should be doing so either in block creation loop or in broadcast

could you clarify what you mean by "standby sending out block in my name?"

Let's say I run two validators A and B, A is B's standby, now I'm switching identity so A becomes the active one, but it's still processing blocks produced by B, in this case do I need to process the block_components for the blocks owned by "me"?

runtime/src/block_component_processor.rs

wen-coding · 2025-11-09T21:58:14Z

runtime/src/block_component_processor.rs

+        if self.has_footer {
+            return Err(BlockComponentProcessorError::MultipleBlockFooters);
+        }
+


Where do we plan to put the parsed content?

wdym by the parsed content?

eventually, on_footer will be the place where process rewards, address the clock, etc.

I mean, the certs in blocks, in the future how do I access it?
self.bank.block_components().... ?

turbine/src/broadcast_stage.rs

wen-coding · 2025-11-09T23:01:59Z

turbine/src/broadcast_stage/broadcast_duplicates_run.rs

            self.num_slots_broadcasted += 1;

-            true
+            self.migration_status.is_alpenglow_enabled()


nit: the above code seems to not be too related to send_header calculation.

it is related, right? we only send headers if alpenglow is enabled

AshwinSekar · 2025-11-10T00:24:25Z

runtime/src/block_component_processor.rs

+}
+
+impl BlockComponentProcessor {
+    pub fn finish(


So rn we have to wait until the block has completed replay in order to do these checks.
A suggestion to make this more online:

add an on_entry_batch which is called right before confirm_slot_entries

confirm_slot has a MigrationStatus available for you to pass in to on_marker / on_entry_batch

If on_entry_batch is called before self.has_header, we immediately know there's no header and can die

If on_entry_batch is called with slot_is_full and !self.has_footer, we immediately know there's no footer and can die

So BlockComponentProcessor processes entry batches and markers online, and we don't need to separately call finish in replay. lmk if that works

this is a great suggestion; in the interest of parallelization, though, I'd like to implement this in a follow-up PR (I promise to get this out today!)

ledger/src/blockstore_processor.rs

AshwinSekar · 2025-11-10T00:32:09Z

runtime/src/block_component_processor.rs

+        Ok(())
+    }
+
+    pub fn on_marker(


Don't feel too strongly about this but for the transaction processor we invoke through bank rather than exposing to blockstore_processor directly, e.g:

alpenglow/runtime/src/bank.rs

Lines 3278 to 3283 in 2a55cd1

let sanitized_output = self

.transaction_processor

.load_and_execute_sanitized_transactions(

self,

sanitized_txs,

check_results,

Could be nice to follow the same style for marker processing, and then blockstore_processor doesn't have to know anything about finding the parent bank or grabbing the write lock.

Makes it easier once we get to implementing the footer logic, by invoking through bank the code might look cleaner for updating accounts and bls verification.

yep, makes sense - could we do this in a follow-up PR rather than in this one?

@AshwinSekar

#### Problem and Summary of Changes Right now, we need to complete replay to verify that `BlockComponent`s in a block satisfy a number of invariants (e.g., exactly one header, exactly one footer, etc.). @AshwinSekar found a clever way to avoid this: #575 (comment). Implement the above.

carllin · 2025-11-10T22:44:26Z

core/src/replay_stage.rs

+                if !is_leader_block {
+                    if let Err(err) = bank
+                        .block_component_processor
+                        .read()
+                        .unwrap()
+                        .finish(migration_status)
+                    {
+                        warn!("Block component processing failed for slot {bank_slot}: {err:?}",);
+                        let root = bank_forks.read().unwrap().root();
+                        Self::mark_dead_slot(
+                            blockstore,
+                            bank,
+                            root,
+                            &BlockstoreProcessorError::BlockComponentProcessor(err),
+                            rpc_subscriptions,
+                            slot_status_notifier,
+                            progress,
+                            duplicate_slots_to_repair,
+                            ancestor_hashes_replay_update_sender,
+                            purge_repair_slot_counter,
+                            &mut tbft_structs,
+                        );
+                        continue;
+                    }
+                }
+


is there any reason not to move this check right after https://github.com/ksn6/alpenglow/blob/abb42d57cf609c64823b53ef39888e7aee001a0b/core/src/replay_stage.rs#L3606-L3639

That way all the failures are directly blocked one after another and don't have to reason about whether logic in between like https://github.com/ksn6/alpenglow/blob/abb42d57cf609c64823b53ef39888e7aee001a0b/core/src/replay_stage.rs#L3675-L3691 is necessary

Could even group the failure checks into a single function check_block_failures

ashwin came up with this nice idea to entirely eliminate the finish() function; we pushed this a few days ago:

#594

We parse `BlockComponent`s in replay and enforce the following checks: - Every block has a block header - Every block has a block footer

@AshwinSekar

…za-xyz#594) Right now, we need to complete replay to verify that `BlockComponent`s in a block satisfy a number of invariants (e.g., exactly one header, exactly one footer, etc.). @AshwinSekar found a clever way to avoid this: anza-xyz#575 (comment). Implement the above.

ksn6 mentioned this pull request Oct 30, 2025

wip, feat: parse BlockComponents in replay #527

Closed

ksn6 force-pushed the parse-block-components-during-replay branch from cac4269 to 02ace98 Compare October 31, 2025 19:05

ksn6 commented Oct 31, 2025

View reviewed changes

local-cluster/tests/local_cluster.rs Outdated Show resolved Hide resolved

ksn6 force-pushed the parse-block-components-during-replay branch from 1543b08 to 5ffab6a Compare November 1, 2025 04:40

ksn6 requested review from AshwinSekar, bw-solana, carllin and wen-coding November 1, 2025 05:20

ksn6 force-pushed the parse-block-components-during-replay branch from fd1c1c9 to 6713039 Compare November 4, 2025 03:21

AshwinSekar reviewed Nov 4, 2025

View reviewed changes

akhi3030 reviewed Nov 6, 2025

View reviewed changes

runtime/src/block_component_verifier.rs Outdated Show resolved Hide resolved

runtime/src/block_component_verifier.rs Outdated Show resolved Hide resolved

runtime/src/block_component_processor.rs Show resolved Hide resolved

ledger/src/blockstore_processor.rs Outdated Show resolved Hide resolved

ksn6 force-pushed the parse-block-components-during-replay branch 4 times, most recently from e6c7b66 to 9b5361d Compare November 7, 2025 05:28

ksn6 enabled auto-merge (squash) November 7, 2025 05:32

ksn6 force-pushed the parse-block-components-during-replay branch 3 times, most recently from e65561a to 12982ff Compare November 8, 2025 21:13

ksn6 force-pushed the parse-block-components-during-replay branch from d7950fa to 2daccd0 Compare November 9, 2025 01:05

ksn6 commented Nov 9, 2025

View reviewed changes

ksn6 changed the title ~~feat: parse BlockComponents during replay~~ feat: "one BlockComponent per CompletedRange" + replay Nov 9, 2025

ksn6 changed the title ~~feat: "one BlockComponent per CompletedRange" + replay~~ feat: enforce "one BlockComponent per CompletedRange" + introduce BlockComponentProcessor Nov 9, 2025

ksn6 changed the title ~~feat: enforce "one BlockComponent per CompletedRange" + introduce BlockComponentProcessor~~ feat: enforce "one BlockComponent per CompletedRange" + replay Nov 9, 2025

wen-coding reviewed Nov 9, 2025

View reviewed changes

AshwinSekar reviewed Nov 10, 2025

View reviewed changes

ksn6 added 2 commits November 10, 2025 12:52

feat: parse BlockComponents during replay

25a46bd

address comments

8dd9ec5

ksn6 added 4 commits November 10, 2025 12:52

address comments

a41fe6b

address comments

1cde4e8

address comments

d85df1f

address comments

abb42d5

ksn6 force-pushed the parse-block-components-during-replay branch from 78afa38 to abb42d5 Compare November 10, 2025 16:52

wen-coding approved these changes Nov 10, 2025

View reviewed changes

ksn6 merged commit 7437655 into anza-xyz:master Nov 10, 2025
7 checks passed

ksn6 deleted the parse-block-components-during-replay branch November 10, 2025 17:35

ksn6 mentioned this pull request Nov 10, 2025

feat: perform all BlockComponent checks in blockstore processor #594

Merged

carllin reviewed Nov 10, 2025

View reviewed changes

AshwinSekar mentioned this pull request Feb 18, 2026

feat(alpenglow): enable blockstore BlockComponent parsing anza-xyz/agave#10566

Merged

ksn6 mentioned this pull request Mar 6, 2026

feat(alpenglow): process block components in blockstore processor anza-xyz/agave#11054

Merged

	let sanitized_output = self
	.transaction_processor
	.load_and_execute_sanitized_transactions(
	self,
	sanitized_txs,
	check_results,

Conversation

ksn6 commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem and Summary of Changes

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

akhi3030 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ksn6 commented Nov 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

ksn6 commented Oct 30, 2025 •

edited

Loading

carllin Nov 10, 2025 •

edited

Loading