💥 Proposal

Currently, the ledger folder contains the following (when using the snarkos start --storage flag`:

1. public ledger data
2. .current-proposal-cache*
3. cached_*_peers
4. jwt_secret_*.txt

Issues:

• if a node operator recovers from a local or remote snapshot, they are likely to overwrite the entire ledger folder
• if you don't use --storage, StorageMode::Production is used, causing more confusingly different behaviour: the .current-proposal-cache* will then be created in the parent directory of the ledger.

Proposal:

• We create a new --storage-local-state flag (name t.b.d.) which stores 2./3./4. , making it explicit these cannot be recovered without backup, and should not be overwritten by an external snapshot
• We ensure both StorageMode::Production and StorageMode::Custom write the appropriate files to this new folder.
• Consider renaming current-proposal-cache to e.g. .current-active-proposal

Design goals:

• Need to manually test all different storage modes
• Backwards compatible?
  • Only make the new flag required for validators from the next release
  • If you don't pass it, log a warning but use the old way of doing things