Skip to content

Commit b611329

Browse files
authored
Daemonize snapshot-converter (#1887)
# Description This PR add the feature for snapshot-converter to run as a daemon. See the help text of ```console $ cabal run snapshot-converter -- --help Utility for converting snapshots among the different snapshot formats used by cardano-node. Usage: snapshot-converter ( (--monitor-lsm-snapshots-in ARG --lsm-database ARG | --monitor-lmdb-snapshots-in ARG) --output-mem-snapshots-in ARG | (--input-lsm-snapshot ARG --input-lsm-database ARG | --input-mem ARG | --input-lmdb ARG) (--output-lsm-snapshot ARG --output-lsm-database ARG | --output-mem ARG | --output-lmdb ARG)) --config PATH [--threshold THRESHOLD] # Running in oneshot mode `snapshot-converter` can be invoked to convert a single snapshot to a different format. The three formats supported at the moment are: Mem, LMDB and LSM. As snapshots in Mem and LMDB are fully contained in one directory, providing that one is enough. On the other hand, converting an LSM snapshot requires a reference to the snapshot directory as well as the LSM database directory. To run in oneshot mode, you have to provide input and output parameters as in: ``` # mem to lsm $ snapshot-converter --input-mem <PATH> --output-lsm-snapshot <PATH> --output-lsm-database <PATH> --config <PATH> # mem to lmdb $ snapshot-converter --input-mem <PATH> --output-lmdb <PATH> --config <PATH> # lmdb to lsm $ snapshot-converter --input-lmdb <PATH> --output-lsm-snapshot <PATH> --output-lsm-database --config <PATH> # lmdb to mem $ snapshot-converter --input-lmdb <PATH> --output-mem <PATH> --config <PATH> # lsm to mem $ snapshot-converter --input-lsm-snapshot <PATH> --input-lsm-database <PATH> --output-mem <PATH> --config <PATH> # lsm to mem $ snapshot-converter --input-lsm-snapshot <PATH> --input-lsm-database <PATH> --output-lmdb <PATH> --config <PATH> ``` Note that the input and output paths need to be named after the slot number of the contained ledger state, this means for example that a snapshot for slot 100 has to be contained in a directory `100[_suffix]` and has to be written to a directory `100[_some_other_suffix]`. Providing a wrong slot number will throw an error. This naming convention is the same expected by `cardano-node`. # Running in daemon mode `snapshot-converter` can be invoked as a daemon to monitor and convert snapshots produced by a `cardano-node` into Mem format as they are written by the node. This is only meaningful to run if your node produces LMDB or LSM snapshots: ``` # lsm to mem $ snapshot-converter --monitor-lsm-snapshots-in <PATH> --lsm-database <PATH> --output-mem-snapshots-in <PATH> --config <PATH> # lmdb to mem $ snapshot-converter --monitor-lmdb-snapshots-in <PATH> --output-mem-snapshots-in <PATH> --config <PATH> ``` Available options: --config PATH Path to config file --threshold THRESHOLD PBftSignatureThreshold -h,--help Show this help text ```
2 parents 7908694 + edc7958 commit b611329

File tree

10 files changed

+836
-631
lines changed

10 files changed

+836
-631
lines changed

ouroboros-consensus-cardano/app/snapshot-converter.hs

Lines changed: 252 additions & 540 deletions
Large diffs are not rendered by default.

ouroboros-consensus-cardano/changelog.d/20260218_112253_javier.sagredo_non_native_snapshots.md

Whitespace-only changes.

ouroboros-consensus-cardano/ouroboros-consensus-cardano.cabal

Lines changed: 39 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,39 @@ library
179179
these ^>=1.2,
180180
validation,
181181

182+
library unstable-snapshot-conversion
183+
import: common-lib
184+
visibility: public
185+
hs-source-dirs: src/unstable-snapshot-conversion
186+
other-modules:
187+
Ouroboros.Consensus.Cardano.StreamingLedgerTables
188+
189+
exposed-modules:
190+
Ouroboros.Consensus.Cardano.SnapshotConversion
191+
192+
build-depends:
193+
ansi-terminal,
194+
base,
195+
cardano-ledger-binary,
196+
cardano-ledger-core,
197+
cardano-ledger-shelley,
198+
cborg,
199+
directory,
200+
filepath,
201+
fs-api,
202+
microlens,
203+
mtl,
204+
ouroboros-consensus:{ouroboros-consensus, ouroboros-consensus-lmdb, ouroboros-consensus-lsm},
205+
ouroboros-consensus-cardano,
206+
random,
207+
resource-registry,
208+
serialise,
209+
sop-core,
210+
sop-extras,
211+
strict-sop-core,
212+
terminal-progress-bar,
213+
text,
214+
182215
library unstable-byronspec
183216
import: common-lib
184217
visibility: public
@@ -692,37 +725,21 @@ executable immdb-server
692725
executable snapshot-converter
693726
import: common-exe
694727
hs-source-dirs: app
695-
other-modules:
696-
Ouroboros.Consensus.Cardano.StreamingLedgerTables
697-
698728
main-is: snapshot-converter.hs
699729
build-depends:
700-
ansi-terminal,
701730
base,
702731
cardano-crypto-class,
703-
cardano-ledger-binary,
704-
cardano-ledger-core,
705-
cardano-ledger-shelley,
706-
cborg,
707-
directory,
708732
filepath,
709-
fs-api,
710-
microlens,
733+
fsnotify,
711734
mtl,
712735
optparse-applicative,
713-
ouroboros-consensus:{ouroboros-consensus, ouroboros-consensus-lmdb, ouroboros-consensus-lsm},
714-
ouroboros-consensus-cardano,
715-
ouroboros-consensus-cardano:unstable-cardano-tools,
716-
random,
717-
resource-registry,
718-
serialise,
719-
sop-core,
720-
sop-extras,
721-
strict-sop-core,
722-
terminal-progress-bar,
723-
text,
736+
ouroboros-consensus,
737+
ouroboros-consensus-cardano:{ouroboros-consensus-cardano, unstable-cardano-tools, unstable-snapshot-conversion},
724738
with-utf8,
725739

740+
if impl(ghc <9.12)
741+
build-depends:
742+
text
726743
other-modules:
727744
DBAnalyser.Parsers
728745

0 commit comments

Comments
 (0)