Skip to content

Commit 51a4915

Browse files
dkijaniadkijania
authored
Merge pull request #17955 from MinaProtocol/dkijania/archive-hardfork-toolbox
[Dry run automation] archive hardfork toolbox
2 parents 3515f45 + 8640694 commit 51a4915

File tree

9 files changed

+620
-0
lines changed

9 files changed

+620
-0
lines changed

‎Makefile‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -221,6 +221,7 @@ build-archive-utils: ocaml_checks reformat-diff ## Build archive node and relate
221221
src/app/archive_blocks/archive_blocks.exe \
222222
src/app/extract_blocks/extract_blocks.exe \
223223
src/app/missing_blocks_auditor/missing_blocks_auditor.exe \
224+
src/app/archive_hardfork_toolbox/archive_hardfork_toolbox.exe \
224225
--profile=$(DUNE_PROFILE) \
225226
&& echo "✅ Build complete"
226227

‎changes/17955.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Archive Hardfork Toolbox
2+
3+
Utility tool for various hardfork verifications or confirmations regarding hardfork runbook automation

‎scripts/debian/builder-helpers.sh‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -714,6 +714,8 @@ copy_common_archive_configs() {
714714
"${BUILDDIR}/usr/local/bin/mina-archive-blocks"
715715
cp ./default/src/app/extract_blocks/extract_blocks.exe \
716716
"${BUILDDIR}/usr/local/bin/mina-extract-blocks"
717+
cp ./default/src/app/archive_hardfork_toolbox/archive_hardfork_toolbox.exe \
718+
"${BUILDDIR}/usr/local/bin/mina-archive-hardfork-toolbox"
717719

718720
mkdir -p "${BUILDDIR}/etc/mina/archive"
719721
cp ../scripts/archive/missing-blocks-guardian.sh \

‎src/app/archive_hardfork_toolbox/README.md‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
# Archive Hardfork Toolbox
2+
3+
The Archive Hardfork Toolbox is a utility for verifying the integrity of archive database migrations and validating hardfork operations in the Mina protocol. This tool helps ensure that database schema upgrades and fork transitions maintain data consistency.
4+
5+
## Overview
6+
7+
This toolbox provides commands to:
8+
- Verify fork block candidates before migration
9+
- Validate database schema upgrades
10+
- Ensure fork block integrity and ancestry
11+
12+
## Commands
13+
14+
### fork-candidate
15+
16+
A group of commands for pre-fork verifications to validate that a candidate block is suitable for forking.
17+
18+
#### is-in-best-chain
19+
20+
Verifies that the fork block is in the best chain of the blockchain.
21+
22+
**Usage:**
23+
```bash
24+
archive_hardfork_toolbox fork-candidate is-in-best-chain \
25+
--postgres-uri "postgresql://user:pass@host:port/db" \
26+
--fork-state-hash "3NKx..." \
27+
--fork-height 12345 \
28+
--fork-slot 67890
29+
```
30+
31+
**Parameters:**
32+
- `--postgres-uri`: URI for connecting to the mainnet archive database
33+
- `--fork-state-hash`: Hash of the fork state
34+
- `--fork-height`: Height of the fork block
35+
- `--fork-slot`: Global slot since genesis of the fork block
36+
37+
#### confirmations
38+
39+
Verifies that the fork block has the required number of confirmations.
40+
41+
**Usage:**
42+
```bash
43+
archive_hardfork_toolbox fork-candidate confirmations \
44+
--postgres-uri "postgresql://user:pass@host:port/db" \
45+
--fork-state-hash "3NKx..." \
46+
--fork-slot 67890 \
47+
--required-confirmations 290
48+
```
49+
50+
**Parameters:**
51+
- `--postgres-uri`: URI for connecting to the mainnet archive database
52+
- `--fork-state-hash`: Hash of the fork state
53+
- `--fork-slot`: Global slot since genesis of the fork block
54+
- `--required-confirmations`: Number of confirmations required for the fork block
55+
56+
#### no-commands-after
57+
58+
Verifies that no commands were executed after the fork block, ensuring a clean fork point.
59+
60+
**Usage:**
61+
```bash
62+
archive_hardfork_toolbox fork-candidate no-commands-after \
63+
--postgres-uri "postgresql://user:pass@host:port/db" \
64+
--fork-state-hash "3NKx..." \
65+
--fork-slot 67890
66+
```
67+
68+
**Parameters:**
69+
- `--postgres-uri`: URI for connecting to the mainnet archive database
70+
- `--fork-state-hash`: Hash of the fork state
71+
- `--fork-slot`: Global slot since genesis of the fork block
72+
73+
### verify-upgrade
74+
75+
Verifies the upgrade from pre-fork to post-fork database schema.
76+
77+
**Usage:**
78+
```bash
79+
archive_hardfork_toolbox verify-upgrade \
80+
--postgres-uri "postgresql://user:pass@host:port/db" \
81+
--version "3.2.0"
82+
```
83+
84+
**Parameters:**
85+
- `--postgres-uri`: URI for connecting to the pre-fork mainnet archive database
86+
- `--version`: Version to upgrade to (e.g., "3.2.0")
87+
88+
### validate-fork
89+
90+
Validates the fork block and its ancestors to ensure blockchain integrity.
91+
92+
**Usage:**
93+
```bash
94+
archive_hardfork_toolbox validate-fork \
95+
--postgres-uri "postgresql://user:pass@host:port/db" \
96+
--fork-state-hash "3NKx..." \
97+
--fork-slot 67890
98+
```
99+
100+
**Parameters:**
101+
- `--postgres-uri`: URI for connecting to the mainnet archive database
102+
- `--fork-state-hash`: Hash of the fork state
103+
- `--fork-slot`: Global slot since genesis of the fork block
104+
105+
## Typical Workflow
106+
107+
1. **Pre-fork validation**: Use the `fork-candidate` commands to ensure the chosen fork point is valid:
108+
- Check if the block is in the best chain
109+
- Verify sufficient confirmations
110+
- Ensure no commands after the fork point
111+
112+
2. **Schema upgrade**: Use `verify-upgrade` to validate the database migration process
113+
114+
3. **Post-fork validation**: Use `validate-fork` to ensure the fork block and its ancestry remain intact
115+
116+
## Database Connection
117+
118+
All commands require an `--postgres-uri` parameter that should be a PostgreSQL connection string in the format:
119+
```
120+
postgresql://username:password@hostname:port/database_name
121+
```

‎src/app/archive_hardfork_toolbox/archive_hardfork_toolbox.ml‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
(* berkeley_migration_verifier.ml -- verify integrity of migrated archive db from original Mina mainnet schema *)
2+
3+
open Async
4+
open Cli_lib.Flag
5+
open Logic
6+
7+
let run_check_and_exit check_fn () =
8+
let open Deferred.Let_syntax in
9+
let%bind results = check_fn () in
10+
report_all_checks results ;
11+
if has_failures results then Shutdown.exit 1 else Deferred.return ()
12+
13+
let fork_state_hash =
14+
Command.Param.(
15+
flag "--fork-state-hash" (required string)
16+
~doc:"String Hash of the fork state")
17+
18+
let fork_slot =
19+
Command.Param.(
20+
flag "--fork-slot" (required int)
21+
~doc:"Int64 Global slot since genesis of the fork block")
22+
23+
let is_in_best_chain_command =
24+
Async.Command.async ~summary:"Verify fork block is in best chain"
25+
(let%map_open.Command { value = postgres_uri; _ } = Uri.Archive.postgres
26+
and fork_state_hash = fork_state_hash
27+
and fork_height =
28+
Command.Param.flag "--fork-height"
29+
Command.Param.(required int)
30+
~doc:"Int Height of the fork block"
31+
and fork_slot = fork_slot in
32+
33+
run_check_and_exit
34+
(is_in_best_chain ~postgres_uri ~fork_state_hash ~fork_height ~fork_slot)
35+
)
36+
37+
let confirmations_command =
38+
Async.Command.async
39+
~summary:"Verify number of confirmations for the fork block"
40+
(let%map_open.Command { value = postgres_uri; _ } = Uri.Archive.postgres
41+
and latest_state_hash =
42+
Command.Param.flag "--latest-state-hash"
43+
Command.Param.(required string)
44+
~doc:"String Hash of the latest state"
45+
and fork_slot = fork_slot
46+
and required_confirmations =
47+
Command.Param.flag "--required-confirmations"
48+
Command.Param.(required int)
49+
~doc:"Int Number of confirmations required for the fork block"
50+
in
51+
52+
run_check_and_exit
53+
(confirmations_check ~postgres_uri ~latest_state_hash
54+
~required_confirmations ~fork_slot ) )
55+
56+
let no_commands_after_command =
57+
Async.Command.async ~summary:"Verify no commands after the fork block"
58+
(let%map_open.Command { value = postgres_uri; _ } = Uri.Archive.postgres
59+
and fork_state_hash = fork_state_hash
60+
and fork_slot = fork_slot in
61+
62+
run_check_and_exit
63+
(no_commands_after ~postgres_uri ~fork_state_hash ~fork_slot) )
64+
65+
let verify_upgrade_command =
66+
Async.Command.async
67+
~summary:"Verify upgrade from pre-fork to post-fork schema"
68+
(let%map_open.Command { value = postgres_uri; _ } = Uri.Archive.postgres
69+
and expected_protocol_version =
70+
Command.Param.flag "--protocol-version"
71+
Command.Param.(required string)
72+
~doc:"String Protocol Version to upgrade to (e.g. 3.2.0 etc)"
73+
and expected_migration_version =
74+
Command.Param.flag "--migration-version"
75+
Command.Param.(required string)
76+
~doc:"String Migration Version that generates current schema"
77+
in
78+
run_check_and_exit
79+
(verify_upgrade ~postgres_uri ~expected_protocol_version
80+
~expected_migration_version ) )
81+
82+
let validate_fork_command =
83+
Async.Command.async ~summary:"Validate fork block and its ancestors"
84+
(let%map_open.Command { value = postgres_uri; _ } = Uri.Archive.postgres
85+
and fork_state_hash = fork_state_hash
86+
and fork_slot = fork_slot in
87+
run_check_and_exit
88+
(validate_fork ~postgres_uri ~fork_state_hash ~fork_slot) )
89+
90+
(* TODO: consider refactor these commands to reuse queries in the future. *)
91+
let commands =
92+
[ ( "fork-candidate"
93+
, Async_command.group ~summary:"Pre-fork verifications"
94+
~preserve_subcommand_order:()
95+
[ ("is-in-best-chain", is_in_best_chain_command)
96+
; ("confirmations", confirmations_command)
97+
; ("no-commands-after", no_commands_after_command)
98+
] )
99+
; ("verify-upgrade", verify_upgrade_command)
100+
; ("validate-fork", validate_fork_command)
101+
]
102+
103+
let () =
104+
Async_command.run
105+
(Async_command.group ~summary:"Archive hardfork toolbox"
106+
~preserve_subcommand_order:() commands )

‎src/app/archive_hardfork_toolbox/dune‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
(executable
2+
(package archive_hardfork_toolbox)
3+
(name archive_hardfork_toolbox)
4+
(public_name archive_hardfork_toolbox)
5+
(libraries
6+
;; opam libraries
7+
async_unix
8+
core
9+
result
10+
async_kernel
11+
uri
12+
stdio
13+
caqti-driver-postgresql
14+
caqti
15+
cli_lib
16+
async
17+
core_kernel
18+
caqti-async
19+
base
20+
base.caml
21+
async.async_command
22+
integers
23+
;; local libraries
24+
logger
25+
archive_lib
26+
block_time
27+
consensus
28+
consensus_vrf
29+
currency
30+
genesis_constants
31+
genesis_ledger_helper
32+
mina_base
33+
mina_base.import
34+
mina_block
35+
mina_caqti
36+
mina_numbers
37+
mina_state
38+
mina_transaction
39+
mina_wire_types
40+
one_or_two
41+
protocol_version
42+
runtime_config
43+
signature_lib
44+
unsigned_extended
45+
with_hash)
46+
47+
(preprocessor_deps ../../config.mlh)
48+
(instrumentation (backend bisect_ppx))
49+
(preprocess (pps ppx_version ppx_mina ppx_let ppx_hash ppx_compare ppx_sexp_conv h_list.ppx)))

0 commit comments

Comments
 (0)