refactor(chain)!: replace `CanonicalIter` with sans-io `CanonicalizationTask` #2038

oleonardolima · 2025-09-18T05:59:57Z

Description

It completes a refactoring of the canonicalization API in bdk_chain, migrating from an iterator-based approach CanonicalIter to a sans-IO/task-based pattern CanonicalizationTask. This change improves the separation of concerns between canonicalization logic and I/O operations, making the code more testable and flexible.

Old API:

// Direct coupling between canonicalization logic and `ChainOracle`
let view = tx_graph.canonical_view(&chain, chain_tip, params)?;

New API:

// Step 1: Create a task (pure logic, no I/O)
let task = tx_graph.canonicalization_task(params);

// Step 2: Execute with a chain oracle (handles I/O)
let view = chain.canonicalize(task, Some(chain_tip));

The new flow works as follows:

Task: CanonicalizationTask encapsulates all canonicalization logic without performing any chain queries.
Query: The task generates CanonicalizationRequests for anchor verification as needed, allowing the ChainOracle to batch or optimize these queries.
Resolution: The ChainOracle (e.g., LocalChain) processes requests and returns CanonicalizationResponse's indicating which anchors are the best in chain.
CanonicalView: Once all queries are resolved, the task builds a complete CanonicalView containing all canonical transactions with their chain positions.

This sans-IO pattern enables:

Tasks can be tested with mock responses without a real chain
Different chain oracles can implement their own optimization strategies
Clear separation between business logic and I/O operations

Notes to the reviewers

The changes are splitted in multiple commits, as I think it could help reviewing it. Also it depends on #2029 PR, as it's built on top of it.

Changelog notice

  ### Changed
  - **Breaking:** Replace `TxGraph::canonical_iter()` and `TxGraph::canonical_view()` with `TxGraph::canonicalization_task()`
  - **Breaking:** Remove `CanonicalIter` in favor of `CanonicalizationTask`
  - **Breaking:** Change canonicalization to use a two-step process: create task, then execute with chain oracle
  - Move `CanonicalReason`, `ObservedIn`, and `CanonicalizationParams` from `canonical_iter` module to `canonical_task` module
  - Add `LocalChain::canonicalize()` method to execute canonicalization tasks

  ### Removed
  - **Breaking:** Remove `canonical_iter` module and `CanonicalIter` type
  - **Breaking:** Remove `TxGraph::try_canonical_view()` and `TxGraph::canonical_view()` methods
  - **Breaking:** Remove `CanonicalView::new()` public constructor

  ### Added
  - New sans-IO `CanonicalizationTask` for stateless canonicalization
  - `CanonicalizationRequest` and `CanonicalizationResponse` types for anchor verification

Checklists

All Submissions:

I followed the contribution guidelines

New Features:

I've added tests for the new feature
I've added docs for the new feature

Bugfixes:

This pull request breaks the existing API
I've added tests to reproduce the issue which are now passing
I'm linking the issue being fixed by this PR

crates/chain/src/canonical_task.rs

evanlinjin · 2025-10-02T06:15:56Z

crates/chain/src/canonical_view.rs

+    /// after completing the canonicalization process. It takes the processed transaction
+    /// data including the canonical ordering, transaction map with chain positions, and
+    /// spend information.
+    pub(crate) fn new(


Nit: I think we can remove this and make all fields pub(crate).

evanlinjin

Great work.

This is my initial round of reviews.

Are you planning to introduce topological ordering in a separate PR?

evanlinjin · 2025-10-02T06:18:01Z

crates/core/src/chain_query.rs

+    ///
+    /// This method processes the response to a previous query request and updates
+    /// the internal state accordingly.
+    fn resolve_query(&mut self, response: ChainResponse<B>);


We should probably mention that:

Queries need to be resolved in order.

The same ChainRequest will be returned from next_query if it's not resolved.

evanlinjin · 2025-10-02T06:43:24Z

crates/bitcoind_rpc/examples/filter_iter.rs

+    let chain_tip = chain.tip().block_id();
+    let task = graph.canonicalization_task(chain_tip, Default::default());
+    let canonical_view = chain.canonicalize(task);


What do you think about the following naming:

CanonicalizationTask -> CanonicalResolver.

TxGraph::canonicalization_task -> TxGraph::resolver.

LocalChain::canonicalize -> LocalChain::resolve.

I've been thinking about this, I agree with the CanonicalResolver, though the TxGraph::resolver and LocalChain::resolve seems a bit off.

What do you think about (?):

CanonicalResolver

TxGraph::canonical_resolver

LocalChain::canonical_resolve

crates/chain/src/canonical_task.rs

crates/chain/src/local_chain.rs

evanlinjin · 2025-10-02T14:27:23Z

crates/chain/src/canonical_task.rs

+            for txid in undo_not_canonical {
+                self.not_canonical.remove(&txid);
+            }
+        } else {


Nit: Have the detected_self_double_spend early return instead of having the else branch.

Rationale: Early return is easier to read and results in less nesting.

I should've addressed this, let me double check.

crates/chain/src/canonical_task.rs

…ionTask` introduces `CanonicalizationTask` that implements canonicalization using a request/response pattern, removing direct dependency on `ChainOracle`. - add `CanonicalizationTask` with request/response pattern for chain queries - track confirmed anchors to eliminate redundant queries - handle direct vs transitive anchor determination - return complete `CanonicalView` with correct chain positions - add `LocalChain::handle_canonicalization_request` helper - export `CanonicalizationTask`, `CanonicalizationRequest`, `CanonicalizationResponse` BREAKING CHANGE: replaces direct `ChainOracle` querying in canonical iteration with a new request/response pattern through `CanonicalizationTask`.

Changes `CanonicalizationRequest` to a struct and `CanonicalizationResponse` to `Option<A>` to process all anchors for a transaction in a single request. - convert `CanonicalizationRequest` from enum to struct with anchors vector - change `CanonicalizationResponse` to `Option<A>` returning best confirmed anchor - batch all anchors for a transaction in one request instead of one-by-one - simplify `process_anchored_txs` to queue all anchors at once - add transitive anchor checking back in `mark_canonical()` This reduces round trips between `CanonicalizationTask` and `ChainOracle` while maintaining the same functionality. The API is cleaner with a struct-based request that mirrors how `scan_anchors` worked in the original `CanonicalIter`. BREAKING CHANGE: `CanonicalizationRequest` and `CanonicalizationResponse` types have changed from enums to struct/type alias respectively.

- Replace `CanonicalView::new()` constructor with internal `CanonicalView::new()` for use by `CanonicalizationTask` - Remove `TxGraph::try_canonical_view()` and `TxGraph::canonical_view()` methods - Add `TxGraph::canonicalization_task()` method to create canonicalization tasks - Add `LocalChain::canonicalize()` method to process tasks and return `CanonicalView`'s - Update `IndexedTxGraph` to delegate canonicalization to underlying `TxGraph` The new API separates canonicalization logic from I/O operations: - Create canonicalization task: `graph.canonicalization_task(params)` - Execute canonicalization: `chain.canonicalize(task, chain_tip)` BREAKING CHANGE: Remove `CanonicalView::new()` and `TxGraph::canonical_view()` methods in favor of task-based approach

- Delete entire `canonical_iter.rs` file and its module declaration - Move `CanonicalReason`, `ObservedIn`, and `CanonicalizationParams` to `canonical_task.rs` - Update module exports to use `pub use canonical_task::*` instead of selective exports BREAKING CHANGE: `CanonicalIter` and all its exports are removed

…icalizationTask` Introduce a new `ChainQuery` trait in `bdk_core` that provides an interface for query-based operations against blockchain data. This trait enables sans-IO patterns for algorithms that need to interact with blockchain oracles without directly performing I/O. The `CanonicalizationTask` now implements this trait, making it more composable and allowing the query pattern to be reused for other blockchain query operations. - Add `ChainQuery` trait with associated types for Request, Response, Context, and Result - Implement `ChainQuery` for `CanonicalizationTask` with `BlockId` as context BREAKING CHANGE: `CanonicalizationTask::finish()` now requires a `BlockId` parameter Co-Authored-By: Claude <[email protected]>

evanlinjin · 2025-10-03T10:40:40Z

crates/core/src/chain_query.rs

+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct ChainRequest<B = BlockId> {
+    /// The chain tip to use as reference for the query.
+    pub chain_tip: B,


I'm wondering whether it will be better to remove this from ChainRequest and have a new method on ChainQuery called something like query_tip(&self) -> BlockId.

Rationale: I'm assuming that we always want this tip to be consistent across the lifetime of the ChainQuery impl.

Counter-argument: We may want to have a ChainQuery impl which queries across conflicting chains - what would this be used for though?

WDYT

crates/chain/src/canonical_task.rs

evanlinjin · 2025-10-07T03:57:39Z

crates/core/src/chain_query.rs

+/// Returns `Some(B)` if at least one of the requested blocks
+/// is confirmed in the chain, or `None` if none are confirmed.
+/// The generic parameter `B` represents the block identifier type, which defaults to `BlockId`.
+pub type ChainResponse<B = BlockId> = Option<B>;


I think it's better API to have:

pub type ChainResponse<B = BlockId> = Vec<(B, bool)>;

Rationale: We only care about whether a block is in the chain or not. However, with Option<B>, we now need to worry about the order the data is coming in.

Counter-argument: This increasing internal complexity.

Counter-counter-argument: No, this decreases internal complexity - allowing us to represent things in the most simplistic manner internally - whether something exists or not.

/// We don't have it yet. This is our `ChainRequest` pending_blocks: BTreeSet<BlockId>, /// We have it! blocks: HashMap<BlockId, bool>,

At the anchored-tx/transitive-tx stages, we try to advance with the blocks we have. If we are missing anything, just populate pending_blocks then try again later. Very simple.

oleonardolima · 2025-10-08T05:08:29Z

Are you planning to introduce topological ordering in a separate PR?

Yes, it would be best to add it after finalizing the API, as it can be done internally.

Make `ChainRequest`/`ChainResponse` generic over block identifier types to enable reuse beyond BlockId. Move `chain_tip` into `ChainRequest` for better encapsulation and simpler API. - Make `ChainRequest` and `ChainResponse` generic types with `BlockId` as default - Add `chain_tip` field to `ChainRequest` to make it self-contained - Change `ChainQuery` trait to use generic parameter `B` for block identifier type - Remove `chain_tip` parameter from `LocalChain::canonicalize()` method - Rename `ChainQuery::Result` to `ChainQuery::Output` for clarity BREAKING CHANGE: - `ChainRequest` now has a `chain_tip` field and is generic over block identifier type - `ChainResponse` is now generic with default type parameter `BlockId` - `ChainQuery` trait now takes a generic parameter `B = BlockId` - `LocalChain::canonicalize()` no longer takes a `chain_tip` parameter Co-authored-by: Claude <[email protected]>

- convert `unprocessed_anchored_txs` from iterator to `VecDeque` - remove `pending_anchor_checks` queue entirely - collect anchored transactions upfront instead of lazy iteration - make `LocalChain::canonicalize()` generic over `ChainQuery` trait Signed-off-by: Leonardo Lima <[email protected]>

…nQuery` Allow any type implementing `ChainQuery` trait instead of requiring `CanonicalizationTask` specifically. Signed-off-by: Leonardo Lima <[email protected]>

- add private `CanonicalStage` enum to track processing phases - add private `try_advance()` for automatic stage progression - add `is_transitive()` helper to `CanonicalReason` - rename internal `confirmed_anchors` to `direct_anchors` for clarity - update `resolve_query()` with stage-specific logic Co-authored-by: Claude <[email protected]> Signed-off-by: Leonardo Lima <[email protected]>

github-project-automation bot added this to BDK Chain Sep 18, 2025

evanlinjin reviewed Sep 18, 2025

View reviewed changes

crates/chain/src/canonical_task.rs Outdated Show resolved Hide resolved

LagginTimes assigned oleonardolima Sep 18, 2025

oleonardolima added this to the Wallet 3.0.0 milestone Sep 18, 2025

notmandatory moved this to In Progress in BDK Chain Sep 18, 2025

oleonardolima mentioned this pull request Sep 19, 2025

refactor(chain)!: remove ChainOracle dependency from CanonicalIter #2034

Draft

6 tasks

oleonardolima force-pushed the refactor/canonical-iter-api branch 6 times, most recently from d851ba6 to c02636d Compare September 23, 2025 00:54

oleonardolima added module-blockchain api A breaking API change labels Sep 23, 2025

oleonardolima force-pushed the refactor/canonical-iter-api branch from c02636d to 78c0538 Compare September 23, 2025 01:08

oleonardolima requested review from LLFourn and LagginTimes September 23, 2025 01:09

oleonardolima mentioned this pull request Sep 24, 2025

refactor(chain)!: update ChainOracle and TxGraph methods to being… #2035

Closed

6 tasks

oleonardolima force-pushed the refactor/canonical-iter-api branch 3 times, most recently from 677e25a to 9e27ab1 Compare September 29, 2025 01:47

evanlinjin reviewed Oct 2, 2025

View reviewed changes

oleonardolima marked this pull request as ready for review October 2, 2025 06:18

evanlinjin reviewed Oct 2, 2025

View reviewed changes

oleonardolima and others added 5 commits October 3, 2025 10:30

oleonardolima force-pushed the refactor/canonical-iter-api branch from 9e27ab1 to f6c8b02 Compare October 3, 2025 00:33

evanlinjin reviewed Oct 7, 2025

View reviewed changes

oleonardolima force-pushed the refactor/canonical-iter-api branch 5 times, most recently from b7f8fba to 4dd96e8 Compare October 8, 2025 04:54

oleonardolima force-pushed the refactor/canonical-iter-api branch from 4dd96e8 to e3282d0 Compare October 8, 2025 05:31

oleonardolima force-pushed the refactor/canonical-iter-api branch from e3282d0 to 065a8d1 Compare October 8, 2025 05:33

oleonardolima and others added 3 commits October 8, 2025 16:34

refactor(chain): make LocalChain::canonicalize() generic over `Chai…

bc2884b

…nQuery` Allow any type implementing `ChainQuery` trait instead of requiring `CanonicalizationTask` specifically. Signed-off-by: Leonardo Lima <[email protected]>

oleonardolima force-pushed the refactor/canonical-iter-api branch from 065a8d1 to 88a99ba Compare October 8, 2025 05:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor(chain)!: replace `CanonicalIter` with sans-io `CanonicalizationTask` #2038

refactor(chain)!: replace `CanonicalIter` with sans-io `CanonicalizationTask` #2038

Uh oh!

oleonardolima commented Sep 18, 2025 •

edited

Loading

Uh oh!

Uh oh!

evanlinjin Oct 2, 2025

Uh oh!

evanlinjin left a comment

Uh oh!

evanlinjin Oct 2, 2025

Uh oh!

evanlinjin Oct 2, 2025

Uh oh!

oleonardolima Oct 8, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

evanlinjin Oct 2, 2025

Uh oh!

oleonardolima Oct 8, 2025

Uh oh!

Uh oh!

evanlinjin Oct 3, 2025

Uh oh!

Uh oh!

evanlinjin Oct 7, 2025

Uh oh!

oleonardolima commented Oct 8, 2025 •

edited

Loading

Uh oh!

Uh oh!

refactor(chain)!: replace CanonicalIter with sans-io CanonicalizationTask #2038

Are you sure you want to change the base?

refactor(chain)!: replace CanonicalIter with sans-io CanonicalizationTask #2038

Uh oh!

Conversation

oleonardolima commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Notes to the reviewers

Changelog notice

Checklists

All Submissions:

New Features:

Bugfixes:

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evanlinjin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oleonardolima commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

refactor(chain)!: replace `CanonicalIter` with sans-io `CanonicalizationTask` #2038

refactor(chain)!: replace `CanonicalIter` with sans-io `CanonicalizationTask` #2038

oleonardolima commented Sep 18, 2025 •

edited

Loading

oleonardolima commented Oct 8, 2025 •

edited

Loading