diff --git a/CIP-0XXX/README.md b/CIP-0XXX/README.md new file mode 100644 index 0000000000..2583d91115 --- /dev/null +++ b/CIP-0XXX/README.md @@ -0,0 +1,297 @@ +--- +CIP: ? +Title: Canonical ledger state snapshot and immutable data formats +Category: Ledger +Status: Proposed +Authors: + - Jean-Philippe Raynaud + - Paul Clark + - TBD +Implementors: + - TBD +Discussions: + - https://github.com/cardano-foundation/CIPs/pull/? +Created: 2025--05-05 +License: CC-BY-4.0 +--- + + + +## Abstract + + + +We propose the adoption of a canonical format for ledger state snapshots and immutable files across all Cardano node implementations. This standardization aims to enable the Mithril protocol to sign ledger state snapshots and facilitate seamless import/export of ledger state and immutable data between different Cardano node implementations. + +## Motivation: why is this CIP necessary? + + + +The Cardano ecosystem is rapidly evolving, with the historical Cardano node and new implementations like Amaru and Acropolis under development. In the future, Stake Pool Operators (SPOs) will likely run diverse node implementations. To ensure seamless interoperability and compatibility of data across these implementations, a standardized approach is essential. This is particularly critical for the Mithril protocol, which depends on consistent signing of ledger state snapshots and immutable files across different nodes. + +Mithril, based on a [Stake-based Threshold Multi-signature scheme](https://iohk.io/en/research/library/papers/mithril-stake-based-threshold-multisignatures/), enables SPOs to certify Cardano chain data in a trustless manner. It is currently used to accelerate full-node bootstrapping and secure light wallets. The protocol aggregates individual signatures from SPOs into multi-signatures and certificates. For Mithril to function securely and effectively, all signer nodes must compute the same message to sign from identical data, and the adoption among SPOs must be very high. + +Currently, certification of the Cardano node database relies on the internal representation of the historical Cardano node database: + +- Immutable files (chunk files and primary/secondary indexes) are deterministically computed by the historical Cardano node. They are signed and served by Mithril. +- Ledger state snapshots, however, are not consistently computed at the same blockchain point across nodes. They are signed using IOG-owned keys instead and they are served by Mithril. + +This limitation means only the historical Cardano node fully benefits from ledger state and immutable file snapshots, and the ledger state snapshot is not yet a first-class citizen in the Mithril protocol. + +Another use case for the ledger state snapshot format is to facilitate debugging and testing of a node implementation. + +## Specification + + + +### Ledger state snapshot format + +#### Requirements + +The ledger state snapshot format should: + +- Be deterministically encoded. +- Support both random access and sequential reading. +- Be adaptable to future changes in the Cardano ecosystem. +- Ensure forward and backward compatibility, allowing new implementations to read older versions and vice versa. +- Be defined using a robust schema notation (e.g., CDDL) to guarantee clarity and validation. + +We propose deterministically encoded CBOR ([RFC8949 Sec 4.2](https://datatracker.ietf.org/doc/html/rfc8949#section-4.2), with the tighter profile defined in [dCBOR](https://datatracker.ietf.org/doc/draft-mcnally-deterministic-cbor/)) with CDDL schemas as the base format and tooling. + +#### Contents + +> [!NOTE] +> +> - Do we want to include derived data in the snapshot? +> - Do we want to create a file per type of data or a single file containing all the data? + +The essential required data in a snapshot include are: + +- UTXOs +- Stake delegation +- Reward accounts +- Protocol parameters +- Accounting pots (reserves, deposits etc.) +- SPO state +- DRep state +- Governance action general state +- Governance action voting state +- Header state (e.g. nonces) + +It could maybe also be useful to produce derived data that is commonly useful: + +- Payment address balances +- Stake address balances +- Stake pool delegation distribution + +##### UTxOs + +```cddl +TODO: Add CDDL schema +``` + +##### Stake delegation + +```cddl +TODO: Add CDDL schema +``` + +##### Reward accounts + +```cddl +TODO: Add CDDL schema +``` + +##### Protocol parameters + +```cddl +TODO: Add CDDL schema +``` + +##### SPO state + +```cddl +TODO: Add CDDL schema +``` + +##### DRep state + +```cddl +TODO: Add CDDL schema +``` + +##### Governance action general state + +```cddl +TODO: Add CDDL schema +``` + +##### Governance action voting state + +```cddl +TODO: Add CDDL schema +``` + +##### Header state + +```cddl +TODO: Add CDDL schema +``` + +##### Payment address balances + +```cddl +TODO: Add CDDL schema +``` + +##### Stake address balances + +```cddl +TODO: Add CDDL schema +``` + +##### Stake pool delegation distribution + +```cddl +TODO: Add CDDL schema +``` + +#### Forward/backward compatibility + +We must ensure that the ledger state snapshot format supports forward and backward compatibility. This guarantees that newer implementations can read older snapshot versions and vice versa without issues. Additionally, Mithril certification must access the ledger state snapshot for a specific version, regardless of the maximum version supported by Cardano nodes. This version would be updated during a new Mithril era, ensuring all Mithril signer nodes transition to the new version simultaneously at an epoch boundary, thereby avoiding split-brain scenarios in the protocol. + +#### Snapshot schedule for Mithril certification + +> [!NOTE] +> Do we want to create a schedule only for Mithril snapshotting purposes, or do we want to create the same schedule for all node implementations? + +Once per epoch, every Cardano node implementation should generate a ledger state snapshot to be signed by Mithril. + +The snapshot should be created at a specific blockchain point, sufficiently distant from epoch boundaries. We propose the following formula to determine this point: + +``` +TODO: Add formula to compute the snapshot point +``` + +The snapshot must reflect the ledger state after processing the specified block number. + +### Ledger state snapshot format + +TODO: Add requirements + +#### Format + +TODO: Add format + +#### Forward/backward compatibility + +TODO: Add forward/backward compatibility + +## Rationale: how does this CIP achieve its goals? + + + +### Why are we proposing this CIP? + +#### For Mithril + +TODO: Add content + +#### For alternative node implementations + +TODO: Add content + +#### For the historical Cardano node + +TODO: Add content + +### For the Cardano ecosystem + +TODO: Add content + +## Path to Active + +### Acceptance Criteria + + + +1. A robust and adaptable ledger state snapshot format that facilitates import/export between node implementations and supports Mithril's signing of ledger state snapshots. +1. A standardized snapshotting schedule for the ledger state snapshot format to enable Mithril's signing process. +1. A flexible and consistent immutable file format for seamless import/export across different node implementations. + +### Implementation Plan + + + +- [ ] Write the specification for the ledger state snapshot format. +- [ ] Write the specification for the ledger state snapshot schedule. +- [ ] Write the specification for the immutable file format. +- [ ] Develop importer/exporter functionality for the ledger state snapshot format: + - [ ] Historical Cardano node + - [ ] Amaru + - [ ] Acropolis + - [ ] Mithril (certification) +- [ ] Develop importer/exporter functionality for the immutable file format: + - [ ] Historical Cardano node + - [ ] Amaru + - [ ] Acropolis + - [ ] Mithril (certification) +- [ ] Implement the snapshotting schedule for the ledger state snapshot format: + - [ ] Historical Cardano node + - [ ] Amaru + - [ ] Acropolis + - [ ] Mithril (certification) + +## References + +### Historical Cardano node + +- **The ChainDB format**: https://cardano-scaling.github.io/cardano-blueprint/storage/chaindb.html +- TODO: Add more references + +### Amaru + +- **Amaru**: https://github.com/pragma-org/amaru +- TODO: Add more references + +### Acropolis + +- **Acropolis**: https://github.com/input-output-hk/acropolis +- TODO: Add more references + +### Mithril + +- **Mithril: Stake-based Threshold Multisignatures**: https://iohk.io/en/research/library/papers/mithril-stake-based-threshold-multisignatures/ +- **Mithril certification**: https://mithril.network/doc/mithril/advanced/mithril-certification/ +- **Mithril certification of Cardano database**: https://mithril.network/doc/mithril/advanced/mithril-certification/cardano-node-database +- **Mithril certification of Cardano database v2**: https://mithril.network/doc/mithril/advanced/mithril-certification/cardano-node-database-v2 +- **Mithril network upgrade strategy**: https://mithril.network/doc/adr/4 +- **Mithril Network Architecture**: https://mithril.network/doc/mithril/mithril-network/architecture +- **Mithril Protocol in depth**: https://mithril.network/doc/mithril/mithril-protocol/protocol +- **Mithril Certificate Chain in depth**: https://mithril.network/doc/mithril/mithril-protocol/certificates +- **Fast Bootstrap a Cardano node**: https://mithril.network/doc/manual/getting-started/bootstrap-cardano-node +- **Run a Mithril Signer node (SPO)**: https://mithril.network/doc/manual/getting-started/run-signer-node/ +- **Mithril Threat Model**: https://mithril.network/doc/mithril/threat-model + +## Copyright + + + +This CIP is licensed under [Apache-2.0](http://www.apache.org/licenses/LICENSE-2.0)