Skip to content

Commit 05117e5

Browse files
tanemtanem
authored
Merge pull request #11 from tanem/docs/snapshot-verification
docs: document snapshot decode/verification
2 parents 73d7479 + ecc66c7 commit 05117e5

4 files changed

Lines changed: 291 additions & 1 deletion

File tree

‎CONTRIBUTING.md‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,15 @@ Before publishing a new version, exercise a real MT5 export from your working tr
3838
3. Store credentials if you haven't already: `uv run mt5-pnl-exporter set-investor-password <login>` and `uv run mt5-pnl-exporter set-encryption-passphrase`.
3939
4. `cp config.example.yaml config.yaml` and fill in `terminal_path` and `accounts`.
4040
5. `uv run mt5-pnl-exporter export` — confirm it logs `OK` per account and writes the snapshot.
41+
6. Verify the snapshot decrypts and validates — this exercises the same `age → gzip → JSON` read path a consumer uses. The on-disk file is ciphertext, so opening it directly won't work; read it back via the API:
4142

42-
Steps 2–5 test the code in your working tree. To also test the **packaged artifact** a consumer installs (entry point, the `[mt5]` extra, the bundled schema file), build and install the wheel before publishing:
43+
```bash
44+
uv run python -c "from pathlib import Path; import mt5_pnl_exporter.snapshot as s, mt5_pnl_exporter.secrets as sec; snap = s.read(Path(r'<snapshot_path>'), sec.get_encryption_passphrase()); print(snap.generated_at, '|', len(snap.closed_deals), 'deals,', len(snap.open_positions), 'open,', len(snap.cash_flows), 'cash flows'); [print(a.login, a.label, a.balance, a.equity) for a in snap.accounts]"
45+
```
46+
47+
Replace `<snapshot_path>` with the value of `snapshot_path` from your `config.yaml`. The `r'...'` raw-string prefix keeps a Windows backslash path (e.g. `Z:\mt5-pnl-exporter\mt5.json.gz.age`) from being mangled by Python escape sequences. If it prints without raising, the file is structurally sound — `read()` reverses the pipeline and validates the full pydantic model.
48+
49+
Steps 2–6 test the code in your working tree. To also test the **packaged artifact** a consumer installs (entry point, the `[mt5]` extra, the bundled schema file), build and install the wheel before publishing:
4350

4451
```bash
4552
uv build # produces dist/*.whl

‎README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,12 @@ MT5 terminal pydantic (~10× smaller) passphrase (atomic .tmp
176176
177177
Gzip + `age` encryption is mandatory, not optional. The on-disk file is always `snapshot.json.gz.age`; readers must reverse the pipeline (`age decrypt → gunzip → json.loads`) to decrypt. Sync services (Dropbox, OneDrive, Syncthing) and backups only ever see ciphertext.
178178
179+
You can't open the file directly — double-clicking a `.age` file just fails. To decode it, use the [age](https://age-encryption.org/) CLI (`brew install age` on macOS; see the age site for other platforms):
180+
181+
```bash
182+
age -d snapshot.json.gz.age | gunzip # prompts for the passphrase, prints the JSON
183+
```
184+
179185
## Schema
180186

181187
`schema/snapshot.schema.json` is generated from the pydantic models and committed. CI (`tests/test_schema_file.py`) fails if it drifts. The on-disk file is the schema's JSON gzipped then encrypted with [age](https://age-encryption.org/) under a passphrase from the OS keychain — consumers must reverse the same pipeline to read it.

‎docs/superpowers/plans/2026-06-10-snapshot-verification-docs.md‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
# Snapshot Verification Docs Implementation Plan
2+
3+
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4+
5+
**Goal:** Give contributors and consumers a copy-pasteable way to decode and verify `snapshot.json.gz.age`, closing the gap where the docs describe the encryption pipeline but show no runnable command.
6+
7+
**Architecture:** Two additive documentation edits, no code/schema/behaviour change. CONTRIBUTING gains a verification step that round-trips a real export through the project's own `snapshot.read()` path; README gains a language-agnostic `age` CLI decode snippet next to the existing pipeline description, with a one-line "it's ciphertext, you can't open it directly" caveat.
8+
9+
**Tech Stack:** Markdown only (`README.md`, `CONTRIBUTING.md`). Verification uses `grep`, the `age` CLI, and the existing `mt5-pnl-exporter` package.
10+
11+
**Spec:** [`docs/superpowers/specs/2026-06-10-snapshot-verification-docs-design.md`](../specs/2026-06-10-snapshot-verification-docs-design.md)
12+
13+
---
14+
15+
## File Structure
16+
17+
- `CONTRIBUTING.md` — append step 6 to "Smoke-test a real export" (after line 40); update the "Steps 2–5" lead-in (line 42) to "Steps 2–6".
18+
- `README.md` — insert the consumer decode snippet immediately after the pipeline sentence in "How it works" (after line 177).
19+
20+
No new files. No `CLAUDE.md` edit (this is not a command/architecture/gotcha change).
21+
22+
---
23+
24+
### Task 1: CONTRIBUTING — add the verification step
25+
26+
**Files:**
27+
- Modify: `CONTRIBUTING.md:40-42`
28+
29+
- [ ] **Step 1: Insert step 6 after the current step 5**
30+
31+
Find this block (lines 40–42):
32+
33+
```markdown
34+
5. `uv run mt5-pnl-exporter export` — confirm it logs `OK` per account and writes the snapshot.
35+
36+
Steps 2–5 test the code in your working tree. To also test the **packaged artifact** a consumer installs (entry point, the `[mt5]` extra, the bundled schema file), build and install the wheel before publishing:
37+
```
38+
39+
Replace it with (adds step 6; changes "Steps 2–5" to "Steps 2–6"):
40+
41+
````markdown
42+
5. `uv run mt5-pnl-exporter export` — confirm it logs `OK` per account and writes the snapshot.
43+
6. Verify the snapshot decrypts and validates — this exercises the same `age → gzip → JSON` read path a consumer uses. The on-disk file is ciphertext, so opening it directly won't work; read it back via the API:
44+
45+
```bash
46+
uv run python -c "from pathlib import Path; import mt5_pnl_exporter.snapshot as s, mt5_pnl_exporter.secrets as sec; snap = s.read(Path('<snapshot_path>'), sec.get_encryption_passphrase()); print(snap.generated_at, '|', len(snap.closed_deals), 'deals,', len(snap.open_positions), 'open,', len(snap.cash_flows), 'cash flows'); [print(a.login, a.label, a.balance, a.equity) for a in snap.accounts]"
47+
```
48+
49+
Replace `<snapshot_path>` with your configured `snapshot_path`. If it prints without raising, the file is structurally sound — `read()` reverses the pipeline and validates the full pydantic model.
50+
51+
Steps 2–6 test the code in your working tree. To also test the **packaged artifact** a consumer installs (entry point, the `[mt5]` extra, the bundled schema file), build and install the wheel before publishing:
52+
````
53+
54+
- [ ] **Step 2: Verify the edit**
55+
56+
Run: `grep -n "Steps 2–6\|Verify the snapshot decrypts\|get_encryption_passphrase" CONTRIBUTING.md`
57+
Expected: three matches — the updated lead-in, the new step-6 heading, and the one-liner. Confirm no remaining "Steps 2–5" in the file: `grep -n "Steps 2–5" CONTRIBUTING.md` returns nothing.
58+
59+
- [ ] **Step 3: Commit**
60+
61+
```bash
62+
git add CONTRIBUTING.md
63+
git commit -m "$(cat <<'EOF'
64+
docs: add snapshot verification step to smoke test
65+
66+
Step 6 reads a freshly-exported snapshot back through snapshot.read(),
67+
confirming the artifact round-trips (not just that export ran) and
68+
exercising the same age → gzip → JSON path a consumer uses.
69+
70+
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
71+
EOF
72+
)"
73+
```
74+
75+
---
76+
77+
### Task 2: README — add the consumer decode snippet
78+
79+
**Files:**
80+
- Modify: `README.md:177`
81+
82+
- [ ] **Step 1: Insert the decode snippet after the pipeline sentence**
83+
84+
Find this line (line 177, in the "How it works" section):
85+
86+
```markdown
87+
Gzip + `age` encryption is mandatory, not optional. The on-disk file is always `snapshot.json.gz.age`; readers must reverse the pipeline (`age decrypt → gunzip → json.loads`) to decrypt. Sync services (Dropbox, OneDrive, Syncthing) and backups only ever see ciphertext.
88+
```
89+
90+
Insert immediately after it (new blank line, then the snippet):
91+
92+
````markdown
93+
The on-disk file is ciphertext — you can't open it directly (double-clicking a `.age` file just fails). To read it, reverse the pipeline. With the [age](https://age-encryption.org/) CLI installed (`brew install age` on macOS; see the age site for other platforms):
94+
95+
```bash
96+
age -d mt5.json.gz.age | gunzip # prompts for the passphrase, prints the JSON
97+
```
98+
````
99+
100+
- [ ] **Step 2: Verify the edit**
101+
102+
Run: `grep -n "age -d mt5.json.gz.age\|can't open it directly\|brew install age" README.md`
103+
Expected: three matches, all within the "How it works" section (immediately after the line ending "only ever see ciphertext").
104+
105+
- [ ] **Step 3: Commit**
106+
107+
```bash
108+
git add README.md
109+
git commit -m "$(cat <<'EOF'
110+
docs: show how to decode the encrypted snapshot
111+
112+
Add a language-agnostic `age -d | gunzip` snippet next to the pipeline
113+
description, with a one-line caveat naming the "can't open .age directly"
114+
dead-end. Install guidance is one example plus the upstream link, not a
115+
per-OS matrix.
116+
117+
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
118+
EOF
119+
)"
120+
```
121+
122+
---
123+
124+
### Task 3: Final verification
125+
126+
**Files:** none (read-only checks).
127+
128+
- [ ] **Step 1: Confirm docs-only — no code/schema touched**
129+
130+
Run: `git diff main --stat`
131+
Expected: only `CONTRIBUTING.md`, `README.md`, and the two spec/plan files under `docs/superpowers/` appear. No files under `src/`, `tests/`, or `schema/`.
132+
133+
- [ ] **Step 2: Confirm the test suite is unaffected**
134+
135+
Run: `uv run pytest -q`
136+
Expected: PASS (docs-only change; nothing in the suite depends on these files).
137+
138+
- [ ] **Step 3 (host-dependent — run where possible, note if skipped): live decode checks**
139+
140+
On the Windows host mid-smoke-test, run the CONTRIBUTING step-6 one-liner against the real snapshot (substituting the real `snapshot_path`):
141+
Expected: prints `generated_at | N deals, N open, N cash flows` then one line per account, without raising.
142+
143+
On a host with the `age` CLI installed (e.g. macOS after `brew install age`), run `age -d mt5.json.gz.age | gunzip`:
144+
Expected: prompts `Enter passphrase:`, then prints the snapshot JSON to stdout.
145+
146+
If either host isn't available in this session, note it as skipped rather than marking it done.
147+
148+
---
149+
150+
## Self-Review
151+
152+
**Spec coverage:**
153+
- "Contributor verification step via `snapshot.read()`" → Task 1. ✓
154+
- "Consumer language-agnostic decode command" → Task 2. ✓
155+
- "One-line ciphertext caveat" → Task 2, step 1. ✓
156+
- "`age -d` no `-p` on decrypt" → Task 2 snippet uses `age -d` with no `-p`. ✓
157+
- "Install guidance = link + one example, not a matrix" → Task 2 snippet. ✓
158+
- "No `CLAUDE.md` change" → File Structure note; Task 3 confirms docs-only. ✓
159+
- "Verification: live decode on Windows + age CLI" → Task 3, step 3. ✓
160+
161+
**Placeholder scan:** `<snapshot_path>` is the only placeholder; it is intentional and the step tells the engineer to substitute the configured `snapshot_path`. No TBD/TODO/"handle edge cases".
162+
163+
**Type consistency:** Function names match the source — `snapshot.read(path, passphrase)` and `secrets.get_encryption_passphrase()` (verified against `src/mt5_pnl_exporter/snapshot.py` and `secrets.py`). `age -d` (no `-p`) is the correct decrypt invocation.

‎docs/superpowers/specs/2026-06-10-snapshot-verification-docs-design.md‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# Snapshot verification docs
2+
3+
Date: 2026-06-10
4+
Status: Approved (design)
5+
6+
## Background
7+
8+
The docs describe the on-disk encryption pipeline but never give a runnable
9+
command to read a snapshot back. `README.md` says readers must reverse the
10+
pipeline (`age decrypt → gunzip → json.loads`) and shows what the decrypted
11+
JSON looks like, but offers no actual command. `CONTRIBUTING.md`'s
12+
"Smoke-test a real export" section stops at step 5 — *"confirm it logs `OK`
13+
per account and writes the snapshot"* — verifying that the export *ran*, never
14+
that the artifact *decrypts to sane contents*.
15+
16+
So the moment a contributor or consumer wants to eyeball the file, they are on
17+
their own. The obvious instinct — double-clicking the `.age` file in Windows
18+
Explorer — dead-ends with "Windows can't open this type of file (.age)",
19+
because the file is ciphertext. This change closes that gap with a
20+
copy-pasteable decode command for each audience.
21+
22+
## Goals
23+
24+
- Give a contributor a verification step that reads a freshly-exported
25+
snapshot back through the project's own read path, confirming the artifact
26+
round-trips — not just that `export` ran.
27+
- Give a consumer (building a reader in any language) a runnable, language-
28+
agnostic decode command next to the existing pipeline description.
29+
- Name the "you can't open the file directly — it's ciphertext" dead-end in one
30+
line, so the reader doesn't repeat the Explorer mistake.
31+
32+
## Non-goals
33+
34+
- No full `model_dump(indent=2)` dump in the docs — useful ad hoc, but noise in
35+
a reference.
36+
- No dedicated Troubleshooting section keyed on the "Windows can't open .age"
37+
symptom — a one-line caveat covers it without the weight.
38+
- No code, schema, or behaviour change. Docs only.
39+
- No `CLAUDE.md` change — this is not a command/architecture/gotcha change, so
40+
the doc-sync rule does not trigger.
41+
42+
## Design decisions
43+
44+
- **Both audiences.** Contributors (verifying a real export) and consumers
45+
(building a reader) each get a decode path.
46+
- **Tailored commands per audience.** CONTRIBUTING uses the Python
47+
`snapshot.read()` one-liner because a contributor has the package installed
48+
and this exercises the project's *actual* validated read path — the real
49+
consumer contract (reverses the pipeline *and* validates the full pydantic
50+
model). README uses the language-agnostic `age` CLI so a consumer in any
51+
language sees how to decode without coupling to internal modules.
52+
- **One-line ciphertext caveat**, not a troubleshooting subsection.
53+
- **`age -d` takes no `-p` on decrypt.** `--passphrase` is an encrypt-only
54+
flag; on decrypt, age auto-detects a passphrase-encrypted (scrypt) file and
55+
prompts. The correct consumer command is `age -d mt5.json.gz.age | gunzip`.
56+
The exporter encrypts via `pyrage`, but the output is standard age format, so
57+
the `age` CLI decrypts it.
58+
59+
## Changes
60+
61+
### A. CONTRIBUTING.md — add step 6 to "Smoke-test a real export"
62+
63+
Append a verification step after the current step 5. Reads the snapshot back
64+
through `snapshot.read()`, which reverses the pipeline and validates the model:
65+
66+
> 6. Verify the snapshot decrypts and validates — this exercises the same
67+
> `age → gzip → JSON` read path a consumer uses. The on-disk file is
68+
> ciphertext, so opening it directly won't work; read it back via the API:
69+
>
70+
> ```bash
71+
> uv run python -c "from pathlib import Path; import mt5_pnl_exporter.snapshot as s, mt5_pnl_exporter.secrets as sec; snap = s.read(Path('<snapshot_path>'), sec.get_encryption_passphrase()); print(snap.generated_at, '|', len(snap.closed_deals), 'deals,', len(snap.open_positions), 'open,', len(snap.cash_flows), 'cash flows'); [print(a.login, a.label, a.balance, a.equity) for a in snap.accounts]"
72+
> ```
73+
>
74+
> Replace `<snapshot_path>` with your configured `snapshot_path`. If it
75+
> prints without raising, the file is structurally sound — `read()` reverses
76+
> the pipeline and validates the full pydantic model.
77+
78+
### B. README.md — decode snippet near the pipeline description
79+
80+
Add a short snippet next to the existing "readers must reverse the pipeline"
81+
text (around the Snapshot-format / decode description). Language-agnostic, with
82+
the one-line ciphertext caveat up front:
83+
84+
> The on-disk file is ciphertext — you can't open it directly (double-clicking
85+
> a `.age` file just fails). Reverse the pipeline to read it. With the
86+
> [age](https://age-encryption.org/) CLI installed (`brew install age` on
87+
> macOS; see the age site for other platforms):
88+
>
89+
> ```bash
90+
> age -d mt5.json.gz.age | gunzip # prompts for the passphrase, prints the JSON
91+
> ```
92+
93+
Install guidance is deliberately one example plus the upstream link, not a
94+
per-OS matrix — the producer host is Windows, consumers run anywhere, and a
95+
full matrix would duplicate upstream and go stale (consistent with the repo's
96+
no-restating-upstream stance). The `age` homepage carries the full matrix.
97+
98+
## Affected files / ripple
99+
100+
- `CONTRIBUTING.md` — new step 6 in "Smoke-test a real export".
101+
- `README.md` — decode snippet + one-line ciphertext caveat near the existing
102+
pipeline description.
103+
- `CLAUDE.md` — no edit; confirm the new wording doesn't contradict the
104+
existing encryption gotchas.
105+
106+
## Verification
107+
108+
- Run the CONTRIBUTING step-6 one-liner against a real snapshot on the Windows
109+
host (live-testable during the current smoke test): it prints the account
110+
summary without raising.
111+
- Run `age -d mt5.json.gz.age | gunzip` on a host with the `age` CLI installed
112+
(e.g. macOS via `brew install age`): it prompts for the passphrase and prints
113+
the JSON.
114+
- No tests, schema, or code touched — `uv run pytest` unaffected (docs-only).

0 commit comments

Comments
 (0)