Skip to content

Bug Fix: Make batch production non-optional during recovery. Add sequencer proof processing tests #2605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
preston-evans98 wants to merge 6 commits into
base: dev
Choose a base branch
from
Open

Bug Fix: Make batch production non-optional during recovery. Add sequencer proof processing tests #2605

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
93c8de4
Add manual proof posting controls for test rollups
preston-evans98 Mar 17, 2026
97e6edc
Improve tests
preston-evans98 Mar 17, 2026
ac85ae7
Fix sequencer recovery batch production bug
preston-evans98 Mar 18, 2026
4ba0f31
Fmt
preston-evans98 Mar 18, 2026
84012ad
Cleanup test code
preston-evans98 Mar 18, 2026
0c93c06
Hopefully fix flake. Add logging on worrying failure
preston-evans98 Mar 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions crates/full-node/sov-sequencer/src/preferred/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,9 +247,7 @@ where
}

self.synchronized_state_updator
.trigger_batch_production_if_convenient_msg(
"recover_and_catch_up:dump_catchup_batches",
)
.trigger_batch_production_msg("recover_and_catch_up:dump_catchup_batches")
Copy link
Member

@bkolad bkolad Mar 19, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the checks that is skipped now is:

if !self.seq_config.automatic_batch_production {
       warn!("Skipping batch production due to settings");
       return;
 }

This means that the sequencer will produce batches during recovery, although it is configured to never do it. This only matters in tests, but it can still be confusing. Let's log an error if we hit recovery and automatic_batch_production = true?

Copy link
Member

@bkolad bkolad Mar 19, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option: we could panic when this happens, and cfg!(debug_assertions) = true

.await
.map_err(|e| e.into_state_update_error())?;
}
Expand Down
8 changes: 5 additions & 3 deletions crates/full-node/sov-sequencer/src/preferred/sync_sequencer_state/inner.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,10 @@ where
return;
}

self.trigger_batch_production().await;
}

pub(crate) async fn trigger_batch_production(&mut self) {
if let Err(e) = self
.try_to_create_and_start_batch_if_none_in_progress(true)
.await
Expand All @@ -414,9 +418,7 @@ where

// If the node is shutting down, we may not be able to terminate the batch. In that case, just return early.
if self.shutdown_receiver.has_changed().unwrap_or(true) {
info!(
"The sequencer is shutting down. Exiting trigger_batch_production_if_convenient."
);
info!("The sequencer is shutting down. Exiting trigger_batch_production.");
return;
}

Expand Down
2 changes: 1 addition & 1 deletion crates/full-node/sov-sequencer/src/preferred/sync_sequencer_state/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ pub(super) enum Message<S: Spec, Rt: Runtime<S>> {
data: SerializedProofWithDetailsBytes,
reason: &'static str,
},
TriggerBatchProductionIfConvenient {
TriggerBatchProduction {
reason: &'static str,
},
SimpleStateUpdate {
Expand Down
10 changes: 4 additions & 6 deletions crates/full-node/sov-sequencer/src/preferred/sync_sequencer_state/sync_state.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -321,9 +321,8 @@ where
data,
reason,
} => self.process_proof_blob(blob_id, data, reason).await,
Message::TriggerBatchProductionIfConvenient { reason } => {
self.process_trigger_batch_production_if_convenient(reason)
.await;
Message::TriggerBatchProduction { reason } => {
self.process_trigger_batch_production(reason).await;
}
Message::SimpleStateUpdate { info } => {
let slot_number = info.slot_number;
Expand Down Expand Up @@ -843,14 +842,13 @@ where
.await;
}

async fn process_trigger_batch_production_if_convenient(&mut self, reason: &'static str) {
async fn process_trigger_batch_production(&mut self, reason: &'static str) {
// We don't run force_overwrite_state() here.
// This is mostly fine, mainly the API state will be out of date until we've
// finished sending our batches.
// Adding parallel state update handling is not worth the complexity right now.

let mut inner = self.get_inner_with_timing(reason).await;
inner.trigger_batch_production_if_convenient().await;
inner.trigger_batch_production().await;
}

async fn process_accept_tx(
Expand Down
5 changes: 2 additions & 3 deletions crates/full-node/sov-sequencer/src/preferred/sync_sequencer_state/updator.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -235,12 +235,11 @@ where
.await
}

pub(crate) async fn trigger_batch_production_if_convenient_msg(
pub(crate) async fn trigger_batch_production_msg(
&self,
reason: &'static str,
) -> Result<(), SequencerStateUpdatorError> {
self.send(Message::TriggerBatchProductionIfConvenient { reason })
.await
self.send(Message::TriggerBatchProduction { reason }).await
}

pub(crate) async fn send_simple_state_update_msg(
Expand Down
Loading
Loading