go/oasis-node: Enable snapshot creation with exact start version

[martintomazic]

Motivation

Storing all historical data for all paratimes from genesis on has exceeded 2TB, hence we are moving towards incremental snapshot creation.

Creating a snapshot with exact start day version is currently practically impossible (e.g. see). Moreover, it takes too much time.

Additional Challenges

1. As the network has no intermediate state checkpoints, we are forced into syncing all the data from the genesis onward which can easily take 2 weeks.
2. Runtime state pruning becomes impractically slow as state grows: Pruning of runtime state is too slow #6334

Proposed Solution

Add new create/import checkpoint commands

Add new oasis-node storage create-checkpoint --version:[uint64] --runtime[hex id], that creates checkpoints for all storage roots of the specified State DB. If --runtime flag is omitted than command creates a checkpoint for the consensus State DB.

To import the checkpoint we could use symmetric oasis-node storage import-checkpoint

Pros

• Trivial to implement, as we already have all building blocks (see checkpoint package).
• Creating a snapshot will no longer require genesis sync, but instead you will start Consensus Block Sync and later followed with Runtime Diff Sync from the exact checkpoint height (!).

Cons

• You need access to a node that has at least as old state as your snapshot start date. Then you have to create a checkpoint on the remote node, copy it to the new node, and finally import it.

Bonus

• Since checkpoints are currently < 10Gb, we could in addition to providing snapshots offer historical checkpoints (e.g. every half a year).
• Somehow related to history expiry: History expiry #6422.

Alternative keep_from pruning configuration.

As suggested by @peternose, probably the cleanest way would be to add keep_from strategy to the pruning config.

This way you could specify height/round from which the data should be preserved.
Assuming you would have pruning enabled from the genesis onward this would work out of the box for you, as internally we could implement very aggressive pruning until this height/round - thread.

Pros

• Relatively easy to implement as this is just another pruner kind for consensus and runtime.

Cons

• You will still have to sync from the genesis onwards.
• Need to extend config, increasing LOC of main oasis logic.
• Update: Add per-runtime pruning config as not every runtime has same version at given consensus height.

We could also implement both.

[peternose]

I would prefer having creation with exact round and not start day. Maybe name oasis-node storage checkpoint create and oasis-node storage checkpoint import .

[martintomazic]

I would prefer having creation with exact round and not start day. Maybe name oasis-node storage checkpoint create and oasis-node storage checkpoint import .

Yes had smtg similar in mind (fixed the title). Probably we will need to use --version terminology if we want same command to work for both consensus and runtimes? Alternative is oasis-node storage consensus/runtime checkpoint create/import --height/round, but then this is 4 commands instead of two which I would prefer to avoid :/.

Update: Nice to have would would be to write/read to/from stdout/stdin.