Validate Database Snapshot

Database Snapshot Validation in a Nutshell

Configure the location of your database:

DBDIR=${DBDIR:-$HOME/.local/share/chainweb/}

Delete Pact database:

rm -rf "$DBDIR"/0/sqlite

Create configuration file for Pact database replay. With this configuration the chainweb-node will not connect to the network.

cat > validate-mainnet-db.yaml <<EOF
databaseDirectory: "$DBDIR"
chainweb:
  # this confirms the Merkle hash of the resulting payload with the respective
  # hash that is stored in the chain database.
  validateHashesOnReplay: true
  p2p:
    peer:
      interface: localhost
      hostaddress:
        port: 4445
    private: true
    ignoreBootstrapNodes: true
  transactionIndex:
    enabled: false
  mempoolP2p:
    enabled: false
  cuts:
    # The following causes full Merkle Tree validation on startup.
    # To only rebuild the Pact db set this to 'none'
    pruneChainDatabase: headers-checked
logging:
  telemetryBackend:
    enabled: false
    configuration:
      color: 'true'
  backend:
    color: 'true'
  filter:
    rules:
    - value: mempool
      key: component
      level: warn
    - value: connection-manager
      key: component
      level: warn
    - value: sync
      key: sub-component
      level: warn
    - value: MempoolAccess
      key: sub-component
      level: warn
    - value: PactService
      key: logger
      level: warn
    - value: Checkpointer
      key: logger
      level: warn
    default: info
EOF

Validate Chainweb database Merkle tree and rebuild Pact database:

chainweb-node --config-file=validate-mainnet-db.yaml

Terminate the chainweb-node when it prints the following log message: finished synchronizing Pact DBs

Details

The Chainweb database consists of two parts:

1. The chain database, which is stored in a rocksdb database and which contains

   • block headers
   • raw block payloads
   • cached block outputs

   All this data is organized in a Merkle DAG which has the headers in the current cut as roots.

2. The Pact database, which is stored in a sqlite database and which contains the Pact state.

There two aspects of validating the consistency of a Chainweb database:

1. Pact history consistency
2. Consistency of the Chain Merkle Tree.

Pact History Replay

On startup, chainweb-node synchronizes the Pact state for each chain to the state of the chain database. For this the system identifies the block which corresponds with the latest Pact state. It then rewinds the Pact state to the largest common ancestor of this block and the most recent block in the chain database of the respective chain. After that is replays that transactions of all blocks up to the most recent block on the respective chain.

An empty or non-existent Pact database corresponds to the genesis header. Therefore, if the Pact database is missing, chainweb-node rebuilds the Pact database from scratch by replaying the transactions of all blocks up to the most recent block.

It is thus possible to rebuild the Pact database of an untrusted database simply by deleting the sqlite directory from the chainweb database directory before starting chainweb-node.

Merkle Tree Validation

Chainweb-node allows to validate the chain Merkle tree while pruning the chainweb database. This can be enabled with the following command line option:

--prune-chain-database=headers-checked=headers-checked

Depending on the system performance validation of the Merkle tree can take anywhere from 30 min to several hours.

Using --prune-chain-database=headers-checked=none completely disables database pruning and Merkle tree validation.