-
Notifications
You must be signed in to change notification settings - Fork 420
feat(rpc)!: Update Emitter::mempool to support evicted_at
#1857
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(rpc)!: Update Emitter::mempool to support evicted_at
#1857
Conversation
771eec8 to
d97eed2
Compare
|
Instead of rebasing, it's probably best to |
f0b807b to
96053d3
Compare
7b25e4a to
16b63c4
Compare
0a20724 feat(examples): Update example crates to use `expected_spk_txids` (志宇) 1f8fc17 feat(core)!: Remove redundant `SyncRequest` methods (志宇) f42d5a8 feat(esplora): Handle spks with expected txids (志宇) 3ab4994 feat(electrum): Handle spks with expected txids (志宇) 64e4100 feat(chain): Add `TxGraph` methods that handle expected spk txids (志宇) b38569f feat(core): Add expected txids to `SyncRequest` spks (Wei Chen) a2dfcb9 feat(chain)!: Change `TxGraph` to understand evicted-at & last-evicted (志宇) ae0336b feat(core): Add `TxUpdate::evicted_ats` (志宇) Pull request description: Partially Fixes #1740. Replaces #1765. Replaces #1811. ### Description This PR allows the receiving structures (`bdk_chain`, `bdk_wallet`) to detect and evict incoming transactions that are double spent (cancelled). We add a new field to `TxUpdate` (`TxUpdate::evicted_ats`), which in turn, updates the `last_evicted` timestamps that are tracked/persisted by `TxGraph`. This is similar to how `TxUpdate::seen_ats` update the `last_seen` timestamp in `TxGraph`. Transactions with a `last_evicted` timestamp higher than their `last_seen` timestamp are considered evicted. `SpkWithExpectedTxids` is introduced in `SpkClient` to track expected `Txid`s for each `spk`. During a sync, if any `Txid`s from `SpkWithExpectedTxids` are not in the current history of an `spk` obtained from the chain source, those `Txid`s are considered missing. Support for `SpkWithExpectedTxids` has been added to both `bdk_electrum` and `bdk_esplora` chain source crates. The canonicalization algorithm is updated to disregard transactions with a `last_evicted` timestamp greater than or equal to their `last_seen` timestamp, except in cases where transitivity rules apply. ### Notes to the reviewers This PR does not fix #1740 for block-by-block chain source (such as `bdk_bitcoind_rpc`). This work is done in a separate PR (#1857). ### Changelog notice * Add `TxUpdate::evicted_ats` which tracks transactions that have been replaced and are no longer present in mempool. * Change `TxGraph` to track `last_evicted` timestamps. This is when a transaction is last marked as missing from the mempool. * The canonicalization algorithm now disregards transactions with a `last_evicted` timestamp greater than or equal to it's `last_seen` timestamp, except when a canonical descendant exists due to rules of transitivity. * Add `SpkWithExpectedTxids` in `spk_client` which keeps track of expected `Txid`s for each `spk`. * Change `bdk_electrum` and `bdk_esplora` to understand `SpkWithExpectedTxids`. * Add `SyncRequestBuilder::expected_txids_of_spk` method which adds an association between `txid`s and `spk`s. ### Checklists #### All Submissions: * [x] I've signed all my commits * [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md) * [x] I ran `cargo fmt` and `cargo clippy` before committing #### New Features: * [x] I've added tests for the new feature * [x] I've added docs for the new feature #### Bugfixes: * [x] This pull request breaks the existing API * [x] I've added tests to reproduce the issue which are now passing * [x] I'm linking the issue being fixed by this PR ACKs for top commit: LLFourn: ACK 0a20724 Tree-SHA512: 29ef964e4597aa9f4cf02e9997b0d17cb91ec2f0f1187b0e9ade3709636b873c2a7cbe803facbc4686a3050a2abeb3e9cc40f9308f8cded9c9353734dcc5755b
16b63c4 to
ca19c92
Compare
8b055d2 to
ca37ae2
Compare
9e00579 to
b94a12a
Compare
evanlinjin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Documentation is outdated, but I understand this is WIP.
b94a12a to
5c21015
Compare
evanlinjin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't finished reviewing yet. Some thoughts:
-
We probably need something like
insert_relevant_evicted_ats(&mut self, evicted_txs: impl IntoIterator<Item = (Txid, u64)>)onTxGraphfor convenience. -
The old best-effort-avoid-reemitting-mempool-txs logic might actually be replaceable by caching
Arc<Transaction>inexpected_mempool_txids.prior_mempool_txs: HashMap<Txid, Arc<Transaction>>,
The idea is we always emit the whole mempool but it is still memory-efficient.
For 2., I guess we can tackle that in a separate PR.
evanlinjin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The logic looks solid.
There’s room for improvement in documentation clarity across tests, inline code comments, and public doc comments:
- Code comments: Focus on the why behind the logic, rather than describing what the code is doing—since that’s usually self-evident from the code itself.
- Tests: For larger tests, consider adding a summary doc comment at the top. This helps readers quickly understand what the test is verifying without having to trace through each line and reconstruct the intent.
- Public doc comments: Prioritize what’s important for the caller to know—avoid going into implementation details unless they directly impact usage.
0c726fb to
1990d22
Compare
1990d22 to
50d30a8
Compare
evanlinjin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, just a final request.
50d30a8 to
ec5262a
Compare
evanlinjin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Apologies for the incremental reviews — it's been eye-opening for me as well how tricky the Emitter is to reason about. Let's aim to simplify it in follow-up PRs.
One thing I’ve come to realize: the next_header method is inconsistent with the eviction-detection logic we're introducing via next_block and mempool. I suggest removing next_header entirely — everything it does can already be achieved through next_block.
ec5262a to
977b4a6
Compare
ccb7a6b to
e1c53d7
Compare
evanlinjin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
self-ACK e1c53d7
|
@LagginTimes Can you rebase on master and also squash 6e1178c into the previous commit? |
* `From<CanonicalTx> for Txid` * `From<CanonicalTx> for Arc<Transaction>` Also added a convenience method `ChainPosition::is_unconfirmed`. These are intended to simplify the calls needed to populate the `expected_mempool_txids` field of `Emitter::new`.
* Change signature of `Emitter::new` so that `expected_mempool_txids` can be more easily constructed from `TxGraph` methods. * Change generic bounds of `C` within `Emitter<C>` to be `C: DeRef, C::Target: RpcApi`. This allows the caller to have `Arc<Client>` as `C` and does not force to caller to hold a lifetimed reference.
e1c53d7 to
8513d83
Compare
ValuedMammal
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK 8513d83
|
Thanks @LagginTimes I didn't mean to become author on 28ef7c9 😅 |
Fixes #1740
Description
Work for this began as part of #1811, based on this comment.
Emitter::mempoolnow returns aMempoolEventwhich provides the data for trackingevicted_at:Changelog notice
Emitter::mempoolto returnMempoolEvents which contain mempool-eviction data.Emitter::clientto have more relaxed generic bounds.C: Deref, C::Target: RpcApiare the new bounds.CanonicalTxtoTxid/Arc<Transaction>.ChainPosition::is_unconfirmedmethod.Checklists
All Submissions:
cargo fmtandcargo clippybefore committingNew Features:
Bugfixes: