Skip to content

Commit d5e4f9a

Browse files
committed
Merge bitcoin/bitcoin#25740: assumeutxo: background validation completion
2b373fe docs: update assumeutxo.md (James O'Beirne) 87a1108 test: add snapshot completion unittests (James O'Beirne) d70919a refactor: make MempoolMutex() public (James O'Beirne) 7300ced log: add LoadBlockIndex() message for assumedvalid blocks (James O'Beirne) d96c59c validation: add ChainMan logic for completing UTXO snapshot validation (James O'Beirne) f2a4f33 move-only-ish: init: factor out chainstate initialization (James O'Beirne) 637a90b add Chainstate::HasCoinsViews() (James O'Beirne) c29f26b validation: add CChainState::m_disabled and ChainMan::isUsable (James O'Beirne) 5ee22cd add ChainstateManager.GetSnapshot{BaseHeight,BaseBlock}() (James O'Beirne) Pull request description: This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: bitcoin/bitcoin#15606) Part two of replacing bitcoin/bitcoin#24232. --- When a user activates a snapshot, the serialized UTXO set data is used to create an "assumed-valid" chainstate, which becomes active in an attempt to get the node to network tip as quickly as possible. Simultaneously in the background, the already-existing chainstate continues "conventional" IBD to both accumulate full block data and serve as a belt-and-suspenders to validate the assumed-valid chainstate. Once the background chainstate's tip reaches the base block of the snapshot used, we set `m_stop_use` on that chainstate and immediately take the hash of its UTXO set; we verify that this matches the assumeutxo value in the source code. Note that while we ultimately want to remove this background chainstate, we don't do so until the following initialization process, when we again check the UTXO set hash of the background chainstate, and if it continues to match, we remove the (now unnecessary) background chainstate, and move the (previously) assumed-valid chainstate into its place. We then reinitialize the chainstate in the normal way. As noted in previous comments, we could do the filesystem operations "inline" immediately when the background validation completes, but that's basically just an optimization that saves disk space until the next restart. It didn't strike me as worth the risk of moving chainstate data around on disk during runtime of the node, though maybe my concerns are overblown. The final result of this completion process is a fully-validated chain, where the only evidence that the user synced using assumeutxo is the existence of a `base_blockhash` file in the `chainstate` directory. ACKs for top commit: achow101: ACK 2b373fe Tree-SHA512: a204e1d6e6932dd83c799af3606b01a9faf893f04e9ee1a36d63f2f1ccfa9118bdc1c107d86976aa0312814267e6a42074bf3e2bf1dead4b2513efc6d955e13d
2 parents fc037c8 + 2b373fe commit d5e4f9a

File tree

5 files changed

+737
-90
lines changed

5 files changed

+737
-90
lines changed

doc/design/assumeutxo.md

Lines changed: 30 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
Assumeutxo is a feature that allows fast bootstrapping of a validating bitcoind
44
instance with a very similar security model to assumevalid.
55

6-
The RPC commands `dumptxoutset` and `loadtxoutset` are used to respectively generate
7-
and load UTXO snapshots. The utility script `./contrib/devtools/utxo_snapshot.sh` may
8-
be of use.
6+
The RPC commands `dumptxoutset` and `loadtxoutset` (yet to be merged) are used to
7+
respectively generate and load UTXO snapshots. The utility script
8+
`./contrib/devtools/utxo_snapshot.sh` may be of use.
99

1010
## General background
1111

@@ -22,10 +22,6 @@ be of use.
2222
chainstate running asynchronously in the background. We also use this flag to control
2323
which index entries are added to setBlockIndexCandidates during LoadBlockIndex().
2424

25-
- Indexing implementations via BaseIndex can no longer assume that indexation happens
26-
sequentially, since background validation chainstates can submit BlockConnected
27-
events out of order with the active chain.
28-
2925
- The concept of UTXO snapshots is treated as an implementation detail that lives
3026
behind the ChainstateManager interface. The external presentation of the changes
3127
required to facilitate the use of UTXO snapshots is the understanding that there are
@@ -76,9 +72,15 @@ original chainstate remains in use as active.
7672

7773
Once the snapshot chainstate is loaded and validated, it is promoted to active
7874
chainstate and a sync to tip begins. A new chainstate directory is created in the
79-
datadir for the snapshot chainstate called `chainstate_snapshot`. When this directory
80-
is present in the datadir, the snapshot chainstate will be detected and loaded as
81-
active on node startup (via `DetectSnapshotChainstate()`).
75+
datadir for the snapshot chainstate called `chainstate_snapshot`.
76+
77+
When this directory is present in the datadir, the snapshot chainstate will be detected
78+
and loaded as active on node startup (via `DetectSnapshotChainstate()`).
79+
80+
A special file is created within that directory, `base_blockhash`, which contains the
81+
serialized `uint256` of the base block of the snapshot. This is used to reinitialize
82+
the snapshot chainstate on subsequent inits. Otherwise, the directory is a normal
83+
leveldb database.
8284

8385
| | |
8486
| ---------- | ----------- |
@@ -88,7 +90,7 @@ active on node startup (via `DetectSnapshotChainstate()`).
8890
The snapshot begins to sync to tip from its base block, technically in parallel with
8991
the original chainstate, but it is given priority during block download and is
9092
allocated most of the cache (see `MaybeRebalanceCaches()` and usages) as our chief
91-
consideration is getting to network tip.
93+
goal is getting to network tip.
9294

9395
**Failure consideration:** if shutdown happens at any point during this phase, both
9496
chainstates will be detected during the next init and the process will resume.
@@ -107,33 +109,32 @@ sequentially.
107109
### Background chainstate hits snapshot base block
108110

109111
Once the tip of the background chainstate hits the base block of the snapshot
110-
chainstate, we stop use of the background chainstate by setting `m_stop_use` (not yet
111-
committed - see #15606), in `CompleteSnapshotValidation()`, which is checked in
112-
`ActivateBestChain()`). We hash the background chainstate's UTXO set contents and
113-
ensure it matches the compiled value in `CMainParams::m_assumeutxo_data`.
114-
115-
The background chainstate data lingers on disk until shutdown, when in
116-
`ChainstateManager::Reset()`, the background chainstate is cleaned up with
117-
`ValidatedSnapshotShutdownCleanup()`, which renames the `chainstate_[hash]` datadir as
118-
`chainstate`.
112+
chainstate, we stop use of the background chainstate by setting `m_disabled`, in
113+
`CompleteSnapshotValidation()`, which is checked in `ActivateBestChain()`). We hash the
114+
background chainstate's UTXO set contents and ensure it matches the compiled value in
115+
`CMainParams::m_assumeutxo_data`.
119116

120117
| | |
121118
| ---------- | ----------- |
122-
| number of chainstates | 2 (ibd has `m_stop_use=true`) |
119+
| number of chainstates | 2 (ibd has `m_disabled=true`) |
123120
| active chainstate | snapshot |
124121

125-
**Failure consideration:** if bitcoind unexpectedly halts after `m_stop_use` is set on
126-
the background chainstate but before `CompleteSnapshotValidation()` can finish, the
127-
need to complete snapshot validation will be detected on subsequent init by
128-
`ChainstateManager::CheckForUncleanShutdown()`.
122+
The background chainstate data lingers on disk until the program is restarted.
129123

130124
### Bitcoind restarts sometime after snapshot validation has completed
131125

132-
When bitcoind initializes again, what began as the snapshot chainstate is now
133-
indistinguishable from a chainstate that has been built from the traditional IBD
134-
process, and will be initialized as such.
126+
After a shutdown and subsequent restart, `LoadChainstate()` cleans up the background
127+
chainstate with `ValidatedSnapshotCleanup()`, which renames the `chainstate_snapshot`
128+
datadir as `chainstate` and removes the now unnecessary background chainstate data.
135129

136130
| | |
137131
| ---------- | ----------- |
138132
| number of chainstates | 1 |
139-
| active chainstate | ibd |
133+
| active chainstate | ibd (was snapshot, but is now fully validated) |
134+
135+
What began as the snapshot chainstate is now indistinguishable from a chainstate that
136+
has been built from the traditional IBD process, and will be initialized as such.
137+
138+
A file will be left in `chainstate/base_blockhash`, which indicates that the
139+
chainstate, even though now fully validated, was originally started from a snapshot
140+
with the corresponding base blockhash.

src/node/chainstate.cpp

Lines changed: 91 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -28,38 +28,13 @@
2828
#include <vector>
2929

3030
namespace node {
31-
ChainstateLoadResult LoadChainstate(ChainstateManager& chainman, const CacheSizes& cache_sizes,
32-
const ChainstateLoadOptions& options)
31+
// Complete initialization of chainstates after the initial call has been made
32+
// to ChainstateManager::InitializeChainstate().
33+
static ChainstateLoadResult CompleteChainstateInitialization(
34+
ChainstateManager& chainman,
35+
const CacheSizes& cache_sizes,
36+
const ChainstateLoadOptions& options) EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
3337
{
34-
auto is_coinsview_empty = [&](Chainstate* chainstate) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) {
35-
return options.reindex || options.reindex_chainstate || chainstate->CoinsTip().GetBestBlock().IsNull();
36-
};
37-
38-
if (!chainman.AssumedValidBlock().IsNull()) {
39-
LogPrintf("Assuming ancestors of block %s have valid signatures.\n", chainman.AssumedValidBlock().GetHex());
40-
} else {
41-
LogPrintf("Validating signatures for all blocks.\n");
42-
}
43-
LogPrintf("Setting nMinimumChainWork=%s\n", chainman.MinimumChainWork().GetHex());
44-
if (chainman.MinimumChainWork() < UintToArith256(chainman.GetConsensus().nMinimumChainWork)) {
45-
LogPrintf("Warning: nMinimumChainWork set below default value of %s\n", chainman.GetConsensus().nMinimumChainWork.GetHex());
46-
}
47-
if (chainman.m_blockman.GetPruneTarget() == std::numeric_limits<uint64_t>::max()) {
48-
LogPrintf("Block pruning enabled. Use RPC call pruneblockchain(height) to manually prune block and undo files.\n");
49-
} else if (chainman.m_blockman.GetPruneTarget()) {
50-
LogPrintf("Prune configured to target %u MiB on disk for block and undo files.\n", chainman.m_blockman.GetPruneTarget() / 1024 / 1024);
51-
}
52-
53-
LOCK(cs_main);
54-
chainman.m_total_coinstip_cache = cache_sizes.coins;
55-
chainman.m_total_coinsdb_cache = cache_sizes.coins_db;
56-
57-
// Load the fully validated chainstate.
58-
chainman.InitializeChainstate(options.mempool);
59-
60-
// Load a chain created from a UTXO snapshot, if any exist.
61-
chainman.DetectSnapshotChainstate(options.mempool);
62-
6338
auto& pblocktree{chainman.m_blockman.m_block_tree_db};
6439
// new CBlockTreeDB tries to delete the existing file, which
6540
// fails if it's still open from the previous loop. Close it first:
@@ -111,6 +86,13 @@ ChainstateLoadResult LoadChainstate(ChainstateManager& chainman, const CacheSize
11186
return {ChainstateLoadStatus::FAILURE, _("Error initializing block database")};
11287
}
11388

89+
auto is_coinsview_empty = [&](Chainstate* chainstate) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) {
90+
return options.reindex || options.reindex_chainstate || chainstate->CoinsTip().GetBestBlock().IsNull();
91+
};
92+
93+
assert(chainman.m_total_coinstip_cache > 0);
94+
assert(chainman.m_total_coinsdb_cache > 0);
95+
11496
// Conservative value which is arbitrarily chosen, as it will ultimately be changed
11597
// by a call to `chainman.MaybeRebalanceCaches()`. We just need to make sure
11698
// that the sum of the two caches (40%) does not exceed the allowable amount
@@ -175,6 +157,84 @@ ChainstateLoadResult LoadChainstate(ChainstateManager& chainman, const CacheSize
175157
return {ChainstateLoadStatus::SUCCESS, {}};
176158
}
177159

160+
ChainstateLoadResult LoadChainstate(ChainstateManager& chainman, const CacheSizes& cache_sizes,
161+
const ChainstateLoadOptions& options)
162+
{
163+
if (!chainman.AssumedValidBlock().IsNull()) {
164+
LogPrintf("Assuming ancestors of block %s have valid signatures.\n", chainman.AssumedValidBlock().GetHex());
165+
} else {
166+
LogPrintf("Validating signatures for all blocks.\n");
167+
}
168+
LogPrintf("Setting nMinimumChainWork=%s\n", chainman.MinimumChainWork().GetHex());
169+
if (chainman.MinimumChainWork() < UintToArith256(chainman.GetConsensus().nMinimumChainWork)) {
170+
LogPrintf("Warning: nMinimumChainWork set below default value of %s\n", chainman.GetConsensus().nMinimumChainWork.GetHex());
171+
}
172+
if (chainman.m_blockman.GetPruneTarget() == std::numeric_limits<uint64_t>::max()) {
173+
LogPrintf("Block pruning enabled. Use RPC call pruneblockchain(height) to manually prune block and undo files.\n");
174+
} else if (chainman.m_blockman.GetPruneTarget()) {
175+
LogPrintf("Prune configured to target %u MiB on disk for block and undo files.\n", chainman.m_blockman.GetPruneTarget() / 1024 / 1024);
176+
}
177+
178+
LOCK(cs_main);
179+
180+
chainman.m_total_coinstip_cache = cache_sizes.coins;
181+
chainman.m_total_coinsdb_cache = cache_sizes.coins_db;
182+
183+
// Load the fully validated chainstate.
184+
chainman.InitializeChainstate(options.mempool);
185+
186+
// Load a chain created from a UTXO snapshot, if any exist.
187+
chainman.DetectSnapshotChainstate(options.mempool);
188+
189+
auto [init_status, init_error] = CompleteChainstateInitialization(chainman, cache_sizes, options);
190+
if (init_status != ChainstateLoadStatus::SUCCESS) {
191+
return {init_status, init_error};
192+
}
193+
194+
// If a snapshot chainstate was fully validated by a background chainstate during
195+
// the last run, detect it here and clean up the now-unneeded background
196+
// chainstate.
197+
//
198+
// Why is this cleanup done here (on subsequent restart) and not just when the
199+
// snapshot is actually validated? Because this entails unusual
200+
// filesystem operations to move leveldb data directories around, and that seems
201+
// too risky to do in the middle of normal runtime.
202+
auto snapshot_completion = chainman.MaybeCompleteSnapshotValidation();
203+
204+
if (snapshot_completion == SnapshotCompletionResult::SKIPPED) {
205+
// do nothing; expected case
206+
} else if (snapshot_completion == SnapshotCompletionResult::SUCCESS) {
207+
LogPrintf("[snapshot] cleaning up unneeded background chainstate, then reinitializing\n");
208+
if (!chainman.ValidatedSnapshotCleanup()) {
209+
AbortNode("Background chainstate cleanup failed unexpectedly.");
210+
}
211+
212+
// Because ValidatedSnapshotCleanup() has torn down chainstates with
213+
// ChainstateManager::ResetChainstates(), reinitialize them here without
214+
// duplicating the blockindex work above.
215+
assert(chainman.GetAll().empty());
216+
assert(!chainman.IsSnapshotActive());
217+
assert(!chainman.IsSnapshotValidated());
218+
219+
chainman.InitializeChainstate(options.mempool);
220+
221+
// A reload of the block index is required to recompute setBlockIndexCandidates
222+
// for the fully validated chainstate.
223+
chainman.ActiveChainstate().UnloadBlockIndex();
224+
225+
auto [init_status, init_error] = CompleteChainstateInitialization(chainman, cache_sizes, options);
226+
if (init_status != ChainstateLoadStatus::SUCCESS) {
227+
return {init_status, init_error};
228+
}
229+
} else {
230+
return {ChainstateLoadStatus::FAILURE, _(
231+
"UTXO snapshot failed to validate. "
232+
"Restart to resume normal initial block download, or try loading a different snapshot.")};
233+
}
234+
235+
return {ChainstateLoadStatus::SUCCESS, {}};
236+
}
237+
178238
ChainstateLoadResult VerifyLoadedChainstate(ChainstateManager& chainman, const ChainstateLoadOptions& options)
179239
{
180240
auto is_coinsview_empty = [&](Chainstate* chainstate) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) {

0 commit comments

Comments
 (0)