Merge #1: The new TxBuilder

c806d5e chore(ci): Make `pin-msrv.sh` exec (志宇)
22ebcfe refactor: Rename `InputStatus` to `TxStatus` (志宇)
0e4a4a6 fix(finalizer): return true if already finalized in `finalize_input` (valued mammal)
ad78206 feat: Add `Selector::select_with_algorithm` (valued mammal)
866f881 feat!: Improve error handling (valued mammal)
1124439 ci: fix typo in rust.yml (valued mammal)
60fa9aa chore: Add examples directory (valued mammal)
386bdae ci: update for rust 1.63.0 (valued mammal)
9bdd0b1 feat!: Remove `bdk_chain` dependency (志宇)
4250de1 feat!: More progress (志宇)
3a5a67e feat!: Replace `PsbtUpdater` and `Builder` with `create_psbt` method (志宇)

Pull request description:

  ## Rationale

  There were many problems with the design of the old `TxBuilder` currently residing in `bdk_wallet`.

  * **No separation of concerns** - All options are crammed into a single `TxParams` struct.
    * Some combinations of parameters are hard to reason about, or may even create invalid transactions.
    * It's difficult to maintain. Adding new features tends to exacerbate the issue by introducing even more variants.
  * **No planning module** - The planning module can help pick the best spending branch just by specifying available assets.
  * **Inaccurate weight calculations** - Coin selection does not currently account for varints of inputs/outputs.
  * "Policies" are applied to all transactions, irrelevant of which inputs were actually chosen.
  * Cannot accurately determine when relatively-locked outputs are spendable.
  * **The "no breaking changes" constraint** This made it difficult to introduce new features to the existing codebase without introducing more footguns.
  * **Coupled to wallet** - Limits flexibility.

  Rewriting the `TxBuilder` and decoupling it from `bdk_wallet` allows us to:

  * Make it usable independently of `bdk_wallet`.
  * Fix all the problems mentioned above cleanly.
  * Avoid waiting for `bdk_wallet v2.0` to ship these fixes/improvements.

  ## Proposed Repository Layout

  I propose turning this repo into a Cargo workspace, containing the following crates:

  * `bdk_tx_core` - This will contain the core `Input`, `InputGroup` types (and maybe more). Both `bdk_chain` and `bdk_wallet` would only need to output these types to interoperate with the rest of the tx-building crates. These types are expected to be relatively stable, with most changes being non-breaking.
  * `bdk_tx` - Handles the coin selection + psbt creation/signing/finalization. @nymius suggested splitting coin selection into its own crate, but I'm not fully convinced that's necessary (for now).
  * `bdk_coin_select` - It would be convenient to move this crate into the workspace for easier development.
  * `bdk_coin_control` - Contains `CoinControl`. This is the only crate in the workspace that depends on `bdk_chain` and `bdk_core`. It handles canonicalization and ensures the caller does not provide a bad set of input candidates (cc. @stevenroose).

  ## Where to Start Reviewing?

  Start with `tests/synopsis.rs` - For now, it's more of an example than a test.

  It includes:
  * A basic create-transaction example.
  * A more involved (but well-commented) cancel-transaction example.

  These examples demonstrate how the various types in the library interact across the different stages of transaction building.

  ## How can I help?

  The main architecture/flow is complete.

  However, there are still a whole bunch of TODOs scattered over the codebase and should be done before merging. These TODOs be summarized as follows (these should be done before merging):

  - [x] Fix up error types. Some methods return `Option`s when they should return custom error enums with variants. It is helpful to know why something failed.
  - [x] Documentation. Things should be better documented. evanlinjin#4
  - [x] Examples. The `tests/synopsis.rs` should be moves to a `examples` folder. The `Wallet` type should be contained in `examples/common/mod.rs` so that it can be shared across examples. evanlinjin#5
  - [x] Update README.

  This is enough to get a merge and get users to start using/testing.

  To help:

  * Review the codebase.
  * Add PRs addressing various TODO comments. The PRs should be made to my fork (`evanlinjin/bdk-tx`).

  ## How long would it take for this to be stable?

  Without any hold-backs, and multiple contributors, I think 2 months is a reasonable time.

ACKs for top commit:
  evanlinjin:
    self-ACK c806d5e

Tree-SHA512: a37fdd2683ed1967912ef2f2828d40a0939d956d8b4372591fcfa3e8e5d458bf478706345ae988d5966efcbf557f1396e4bb8f6f98c0dedc891b5cb790c97e77

Author: evanlinjin

.github/workflows/rust.yml

@@ -14,14 +14,17 @@ jobs:
     strategy:
       matrix:
         rust:
-          - toolchain: stable
-          - toolchain: 1.63.0
+          - version: stable
+          - version: 1.63.0
     steps:
       - uses: actions/checkout@v4
       - name: Install Rust
         uses: dtolnay/rust-toolchain@v1
         with:
-            toolchain: ${{ matrix.rust.toolchain }}
+            toolchain: ${{ matrix.rust.version }}
+      - name: Pin dependencies for MSRV
+        if: matrix.rust.version == '1.63.0'
+        run: ./ci/pin-msrv.sh
       - name: Test
         run: cargo test --no-fail-fast --all-features
 

Cargo.toml

@@ -10,13 +10,23 @@ readme = "README.md"
 
 [dependencies]
 miniscript = { version = "12", default-features = false }
+bdk_coin_select = "0.4.0"
 
 [dev-dependencies]
 anyhow = "1"
-bdk_chain = { version = "0.21" }
 bdk_tx = { path = "." }
 bitcoin = { version = "0.32", features = ["rand-std"] }
+bdk_testenv = "0.11.1"
+bdk_bitcoind_rpc = "0.18.0"
+bdk_chain = { version = "0.21" }
 
 [features]
 default = ["std"]
 std = ["miniscript/std"]
+
+[[example]]
+name = "synopsis"
+
+[[example]]
+name = "common"
+crate-type = ["lib"]

README.md

@@ -1,4 +1,4 @@
-# `bdk-tx`
+# `bdk_tx`
 
 This is a transaction building library based on `rust-miniscript` that lets you build, update, and finalize PSBTs with minimal dependencies.
 
@@ -7,26 +7,9 @@ Because the project builds upon [miniscript] we support [descriptors] natively.
 
 Refer to [BIP174], [BIP370], and [BIP371] to learn more about partially signed bitcoin transactions (PSBT).
 
-## Example
+**Note:**
+The library is unstable and API changes should be expected. Check the [examples] directory for detailed usage examples.
 
-To get started see the `DataProvider` trait and the methods for adding inputs and outputs.
-
-```rust
-use bdk_tx::Builder;
-use bdk_tx::DataProvider;
-
-impl DataProvider for MyType { ... }
-
-let mut builder = Builder::new();
-builder.add_input(plan_utxo);
-builder.add_output(script_pubkey, amount);
-let (mut psbt, finalizer) = builder.build_tx(data_provider)?;
-
-// Your PSBT signing flow...
-
-let result = finalizer.finalize(&mut psbt)?;
-assert!(result.is_finalized());
-```
 
 ## Contributing
 Found a bug, have an issue or a feature request? Feel free to open an issue on GitHub. This library is open source licensed under MIT.
@@ -36,3 +19,4 @@ Found a bug, have an issue or a feature request? Feel free to open an issue on G
 [BIP174]: https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki
 [BIP370]: https://github.com/bitcoin/bips/blob/master/bip-0370.mediawiki
 [BIP371]: https://github.com/bitcoin/bips/blob/master/bip-0371.mediawiki
+[examples]: ./examples

ci/pin-msrv.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+set -x
+set -euo pipefail
+
+# Script to pin dependencies for MSRV
+
+# cargo clean
+
+# rm -f Cargo.lock
+
+# rustup default 1.63.0
+
+cargo update -p zstd-sys --precise "2.0.8+zstd.1.5.5"
+cargo update -p time --precise "0.3.20"
+cargo update -p home --precise "0.5.5"
+cargo update -p flate2 --precise "1.0.35"
+cargo update -p once_cell --precise "1.20.3"
+cargo update -p bzip2-sys --precise "0.1.12"
+cargo update -p ring --precise "0.17.12"
+cargo update -p once_cell --precise "1.20.3"
+cargo update -p base64ct --precise "1.6.0"
+cargo update -p minreq --precise "2.13.2"

examples/common.rs

@@ -0,0 +1,170 @@
+use std::sync::Arc;
+
+use bdk_bitcoind_rpc::Emitter;
+use bdk_chain::{bdk_core, Anchor, Balance, ChainPosition, ConfirmationBlockTime};
+use bdk_testenv::{bitcoincore_rpc::RpcApi, TestEnv};
+use bdk_tx::{CanonicalUnspents, Input, InputCandidates, RbfParams, TxStatus, TxWithStatus};
+use bitcoin::{absolute, Address, BlockHash, OutPoint, Transaction, Txid};
+use miniscript::{
+    plan::{Assets, Plan},
+    Descriptor, DescriptorPublicKey, ForEachKey,
+};
+
+const EXTERNAL: &str = "external";
+const INTERNAL: &str = "internal";
+
+pub struct Wallet {
+    pub chain: bdk_chain::local_chain::LocalChain,
+    pub graph: bdk_chain::IndexedTxGraph<
+        bdk_core::ConfirmationBlockTime,
+        bdk_chain::keychain_txout::KeychainTxOutIndex<&'static str>,
+    >,
+}
+
+impl Wallet {
+    pub fn new(
+        genesis_hash: BlockHash,
+        external: Descriptor<DescriptorPublicKey>,
+        internal: Descriptor<DescriptorPublicKey>,
+    ) -> anyhow::Result<Self> {
+        let mut indexer = bdk_chain::keychain_txout::KeychainTxOutIndex::default();
+        indexer.insert_descriptor(EXTERNAL, external)?;
+        indexer.insert_descriptor(INTERNAL, internal)?;
+        let graph = bdk_chain::IndexedTxGraph::new(indexer);
+        let (chain, _) = bdk_chain::local_chain::LocalChain::from_genesis_hash(genesis_hash);
+        Ok(Self { chain, graph })
+    }
+
+    pub fn sync(&mut self, env: &TestEnv) -> anyhow::Result<()> {
+        let client = env.rpc_client();
+        let last_cp = self.chain.tip();
+        let mut emitter = Emitter::new(client, last_cp, 0);
+        while let Some(event) = emitter.next_block()? {
+            let _ = self
+                .graph
+                .apply_block_relevant(&event.block, event.block_height());
+            let _ = self.chain.apply_update(event.checkpoint);
+        }
+        let mempool = emitter.mempool()?;
+        let _ = self.graph.batch_insert_relevant_unconfirmed(mempool);
+        Ok(())
+    }
+
+    pub fn next_address(&mut self) -> Option<Address> {
+        let ((_, spk), _) = self.graph.index.next_unused_spk(EXTERNAL)?;
+        Address::from_script(&spk, bitcoin::consensus::Params::REGTEST).ok()
+    }
+
+    pub fn balance(&self) -> Balance {
+        let outpoints = self.graph.index.outpoints().clone();
+        self.graph.graph().balance(
+            &self.chain,
+            self.chain.tip().block_id(),
+            outpoints,
+            |_, _| true,
+        )
+    }
+
+    /// TODO: Add to chain sources.
+    pub fn tip_info(
+        &self,
+        client: &impl RpcApi,
+    ) -> anyhow::Result<(absolute::Height, absolute::Time)> {
+        let tip = self.chain.tip().block_id();
+        let tip_info = client.get_block_header_info(&tip.hash)?;
+        let tip_height = absolute::Height::from_consensus(tip.height)?;
+        let tip_time =
+            absolute::Time::from_consensus(tip_info.median_time.unwrap_or(tip_info.time) as _)?;
+        Ok((tip_height, tip_time))
+    }
+
+    // TODO: Maybe create an `AssetsBuilder` or `AssetsExt` that makes it easier to add
+    // assets from descriptors, etc.
+    pub fn assets(&self) -> Assets {
+        let index = &self.graph.index;
+        let tip = self.chain.tip().block_id();
+        Assets::new()
+            .after(absolute::LockTime::from_height(tip.height).expect("must be valid height"))
+            .add({
+                let mut pks = vec![];
+                for (_, desc) in index.keychains() {
+                    desc.for_each_key(|k| {
+                        pks.extend(k.clone().into_single_keys());
+                        true
+                    });
+                }
+                pks
+            })
+    }
+
+    pub fn plan_of_output(&self, outpoint: OutPoint, assets: &Assets) -> Option<Plan> {
+        let index = &self.graph.index;
+        let ((k, i), _txout) = index.txout(outpoint)?;
+        let desc = index.get_descriptor(k)?.at_derivation_index(i).ok()?;
+        let plan = desc.plan(assets).ok()?;
+        Some(plan)
+    }
+
+    pub fn canonical_txs(&self) -> impl Iterator<Item = TxWithStatus<Arc<Transaction>>> + '_ {
+        pub fn status_from_position(pos: ChainPosition<ConfirmationBlockTime>) -> Option<TxStatus> {
+            match pos {
+                bdk_chain::ChainPosition::Confirmed { anchor, .. } => Some(TxStatus {
+                    height: absolute::Height::from_consensus(
+                        anchor.confirmation_height_upper_bound(),
+                    )
+                    .expect("must convert to height"),
+                    time: absolute::Time::from_consensus(anchor.confirmation_time as _)
+                        .expect("must convert from time"),
+                }),
+                bdk_chain::ChainPosition::Unconfirmed { .. } => None,
+            }
+        }
+        self.graph
+            .graph()
+            .list_canonical_txs(&self.chain, self.chain.tip().block_id())
+            .map(|c_tx| (c_tx.tx_node.tx, status_from_position(c_tx.chain_position)))
+    }
+
+    pub fn all_candidates(&self) -> bdk_tx::InputCandidates {
+        let index = &self.graph.index;
+        let assets = self.assets();
+        let canon_utxos = CanonicalUnspents::new(self.canonical_txs());
+        let can_select = canon_utxos.try_get_unspents(
+            index
+                .outpoints()
+                .iter()
+                .filter_map(|(_, op)| Some((*op, self.plan_of_output(*op, &assets)?))),
+        );
+        InputCandidates::new([], can_select)
+    }
+
+    pub fn rbf_candidates(
+        &self,
+        replace: impl IntoIterator<Item = Txid>,
+        tip_height: absolute::Height,
+    ) -> anyhow::Result<(bdk_tx::InputCandidates, RbfParams)> {
+        let index = &self.graph.index;
+        let assets = self.assets();
+        let mut canon_utxos = CanonicalUnspents::new(self.canonical_txs());
+
+        // Exclude txs that reside-in `rbf_set`.
+        let rbf_set = canon_utxos.extract_replacements(replace)?;
+        let must_select = rbf_set
+            .must_select_largest_input_of_each_original_tx(&canon_utxos)?
+            .into_iter()
+            .map(|op| canon_utxos.try_get_unspent(op, self.plan_of_output(op, &assets)?))
+            .collect::<Option<Vec<Input>>>()
+            .ok_or(anyhow::anyhow!(
+                "failed to find input of tx we are intending to replace"
+            ))?;
+
+        let can_select = index.outpoints().iter().filter_map(|(_, op)| {
+            canon_utxos.try_get_unspent(*op, self.plan_of_output(*op, &assets)?)
+        });
+        Ok((
+            InputCandidates::new(must_select, can_select)
+                .filter(rbf_set.candidate_filter(tip_height)),
+            rbf_set.selector_rbf_params(),
+        ))
+    }
+}