Skip to content

Change CanonicalState sharing model.#5437

Closed
deuszx wants to merge 4 commits intotestnet_conwayfrom
deuszx/rm-invalid-caching-layer
Closed

Change CanonicalState sharing model.#5437
deuszx wants to merge 4 commits intotestnet_conwayfrom
deuszx/rm-invalid-caching-layer

Conversation

@deuszx
Copy link
Copy Markdown
Contributor

@deuszx deuszx commented Feb 13, 2026

Motivation

The exporter's CanonicalState is cloned via clone_unchecked(), giving the exporter a LogView with a frozen stored_count. After flush persists entries to DB, the exporter's LogView still can't read them – LogView::get() routes index >= stored_count to new_values (empty in the clone), never hitting the DB. This means the in-memory buffer can never be cleared and grows without bound.

Proposal

Share a single CanonicalState behind Arc<tokio::sync::RwLock> instead of cloning it. Both the block processor and exporter use the same LogView, so stored_count stays consistent. After flush +post_save(), the buffer is safely drained — subsequent reads fall through to the LogView which has the correct stored_count.

CanonicalState

  • Removed flushed_count field (flush now drains the entire buffer)
  • Replaced Arc<papaya::HashMap<usize, CanonicalBlock>> with Vec<CanonicalBlock> (RwLock provides concurrency; entries are sequential)
  • Removed clone() (no more clone_unchecked()) and next_index()
  • flush() now uses buffer.drain(..) instead of iterating flushed_count..count

SharedStorage

  • shared_canonical_state field is now Arc<tokio::sync::RwLock<CanonicalState<C>>>
  • clone() is infallible (Arc clone) — kept Result return type to avoid touching 13+ call sites
  • push_block() is now async (acquires write lock)

ExporterStorage / BlockProcessorStorage

  • Read operations (get_block_with_blob_ids, get_block_with_blobs, get_latest_index) acquire a read lock
  • Write operations (push_block, save) acquire a write lock
  • clone() takes &self instead of &mut self

Test Plan

  • cargo build -p linera-service — compiles with no errors
  • cargo test -p linera-service --bin linera-exporter — all existing tests pass (including test_committee_destination)

Release Plan

  • These changes should be released to testnet, and then
  • Backported to main.

Links

…flush

Replace clone_unchecked() sharing with a single CanonicalState behind
Arc<tokio::sync::RwLock>.  Both the block processor and exporter now
use the same LogView, so stored_count stays consistent after
flush + post_save and the buffer can be safely drained.
@deuszx
Copy link
Copy Markdown
Contributor Author

deuszx commented Feb 16, 2026

Superseeded by #5449

@deuszx deuszx closed this Feb 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant