|
| 1 | +# Phase 1b cycle 2: mandatory age encryption |
| 2 | + |
| 3 | +Status: design. Approved 2026-06-01, ready to plan implementation. |
| 4 | + |
| 5 | +Refines cycle 2 of [`2026-06-01-phase-1b-design.md`](2026-06-01-phase-1b-design.md) |
| 6 | +(item 6) and folds in the gzip-before-encryption transport wrapper that |
| 7 | +[`2026-06-01-phase-1b-cycle-1-design.md`](2026-06-01-phase-1b-cycle-1-design.md) |
| 8 | +flagged as a forward-looking task. |
| 9 | + |
| 10 | +## Why |
| 11 | + |
| 12 | +The cycle 1 snapshot is plaintext JSON. Users who move it through a |
| 13 | +third-party sync service (Dropbox, OneDrive, Syncthing) leave their full |
| 14 | +trading history at rest on someone else's disk. Encrypting at rest closes |
| 15 | +that gap. age is small, well-specified, and has multiple independent |
| 16 | +implementations — a reasonable contract to commit to in 1.0. |
| 17 | + |
| 18 | +Gzip lands in the same cycle because both belong at the transport layer |
| 19 | +and both touch `snapshot.write` / `snapshot.read`. Compressing JSON-with- |
| 20 | +repeated-field-names 5–10× makes the encrypted file viable on the same |
| 21 | +sync services we're encrypting for. |
| 22 | + |
| 23 | +## Wire format |
| 24 | + |
| 25 | +The pipeline, applied left-to-right on write and right-to-left on read: |
| 26 | + |
| 27 | +``` |
| 28 | +Snapshot model → JSON bytes → gzip → age (passphrase) → file |
| 29 | +``` |
| 30 | + |
| 31 | +File extension by convention: `snapshot.json.gz.age`. The exporter writes |
| 32 | +to `cfg.snapshot_path` verbatim — no auto-suffix, no enforcement. |
| 33 | +`config.example.yaml` updates to the new extension so the convention is |
| 34 | +documented. Consumers point at the same path the exporter writes; they |
| 35 | +must implement the same pipeline in reverse to read it. |
| 36 | + |
| 37 | +## Producer |
| 38 | + |
| 39 | +`snapshot.write(path, snap, passphrase)` becomes the only write API. |
| 40 | +Atomic-write semantics from cycle 1 are preserved — the temp file holds |
| 41 | +the encrypted bytes. |
| 42 | + |
| 43 | +```python |
| 44 | +data = snap.model_dump_json(indent=2).encode() |
| 45 | +compressed = gzip.compress(data) |
| 46 | +encrypted = pyrage.passphrase.encrypt(compressed, passphrase) |
| 47 | +tmp.write_bytes(encrypted) |
| 48 | +tmp.replace(path) |
| 49 | +``` |
| 50 | + |
| 51 | +`poll` retrieves the passphrase from the keychain before touching MT5; if |
| 52 | +missing, it exits before any work happens (see below). |
| 53 | + |
| 54 | +## Consumer |
| 55 | + |
| 56 | +`snapshot.read(path, passphrase)` becomes the only read API. |
| 57 | + |
| 58 | +```python |
| 59 | +encrypted = path.read_bytes() |
| 60 | +compressed = pyrage.passphrase.decrypt(encrypted, passphrase) |
| 61 | +data = gzip.decompress(compressed) |
| 62 | +raw = json.loads(data) |
| 63 | +# existing schema-version check + Snapshot.model_validate |
| 64 | +``` |
| 65 | + |
| 66 | +Decryption errors (`pyrage` exceptions) surface as `ValueError` with a |
| 67 | +message naming the likely cause — wrong passphrase or corrupt file. |
| 68 | +gzip errors after a successful decrypt are treated the same way (corrupt |
| 69 | +or tampered payload). |
| 70 | + |
| 71 | +Schema-version check and `Snapshot.model_validate` run on the decrypted |
| 72 | +JSON exactly as today. |
| 73 | + |
| 74 | +## Passphrase storage |
| 75 | + |
| 76 | +Keychain under the existing `KEYRING_SERVICE = "mt5-pnl-exporter"`, with |
| 77 | +account `"encryption-passphrase"`. No collision with login-keyed entries |
| 78 | +(those use integer-string accounts). |
| 79 | + |
| 80 | +`secrets.py` gains: |
| 81 | + |
| 82 | +```python |
| 83 | +ENCRYPTION_PASSPHRASE_ACCOUNT = "encryption-passphrase" |
| 84 | + |
| 85 | +def get_encryption_passphrase() -> str | None: ... |
| 86 | +def set_encryption_passphrase(passphrase: str) -> None: ... |
| 87 | +``` |
| 88 | + |
| 89 | +Same `redact_filter` register pattern as investor passwords: on |
| 90 | +retrieval, register with the filter so it's stripped from any log line. |
| 91 | + |
| 92 | +## `set-encryption-passphrase` command |
| 93 | + |
| 94 | +Mirrors `set-password`: |
| 95 | + |
| 96 | +```bash |
| 97 | +mt5-pnl-exporter set-encryption-passphrase |
| 98 | +``` |
| 99 | + |
| 100 | +- No login argument — one passphrase per host. |
| 101 | +- Prompts twice via `getpass` (entry + confirmation) and refuses on |
| 102 | + mismatch. Refuses empty input. |
| 103 | +- On success, stores in keychain and prints a green confirmation to |
| 104 | + stderr. |
| 105 | + |
| 106 | +## Missing-passphrase behaviour |
| 107 | + |
| 108 | +Mandatory means: no passphrase, no work. `poll` checks the keychain |
| 109 | +before any MT5 connection. On absence, exits 1 with this message to |
| 110 | +stderr (literal text, used in tests): |
| 111 | + |
| 112 | +``` |
| 113 | +Error: no encryption passphrase set in keychain. |
| 114 | +Run 'mt5-pnl-exporter set-encryption-passphrase' first. |
| 115 | +``` |
| 116 | + |
| 117 | +`snapshot.read()` raises `RuntimeError` with the same message body if |
| 118 | +called with `passphrase=None`. Consumers that surface a friendlier |
| 119 | +message can catch it. |
| 120 | + |
| 121 | +No config flag, no `--no-encrypt` escape hatch, no two code paths. One |
| 122 | +pipeline. |
| 123 | + |
| 124 | +## Tests |
| 125 | + |
| 126 | +- `test_snapshot.py` round-trip extended: write → read with passphrase, |
| 127 | + equality across all four record types. |
| 128 | +- Wrong-passphrase read raises `ValueError` matching "wrong passphrase or |
| 129 | + corrupt file". |
| 130 | +- Corrupt encrypted bytes (truncate the file) raise the same `ValueError`. |
| 131 | +- Schema-version mismatch still detected after decrypt (existing test |
| 132 | + generalised — encrypt a v2-tagged blob, then tamper the inner JSON to |
| 133 | + v999, expect rejection). |
| 134 | +- `snapshot.read(path, passphrase=None)` raises `RuntimeError` matching |
| 135 | + the documented missing-passphrase message. |
| 136 | +- `test_secrets.py`: `set_encryption_passphrase` round-trips through the |
| 137 | + in-memory keyring fixture; `get_encryption_passphrase` returns `None` |
| 138 | + when unset. |
| 139 | +- `test_cli.py`: |
| 140 | + - `poll` exits 1 with the documented stderr message when the |
| 141 | + encryption passphrase is missing — and does NOT touch the fake |
| 142 | + `MT5Source` (the in-test fake records calls; assertion: zero). |
| 143 | + - `poll` happy path: encryption passphrase fixtured into the in-memory |
| 144 | + keyring; verify the on-disk file decrypts cleanly with the same |
| 145 | + passphrase. |
| 146 | + - `set-encryption-passphrase`: empty refused; mismatched confirmation |
| 147 | + refused; success stores in keychain. |
| 148 | +- The cycle 1 `tests/fixtures/sample_snapshot.json` stays as the plaintext |
| 149 | + reference. The encrypted file used in tests is generated in-test from |
| 150 | + the JSON fixture + a fixed test passphrase so it stays reproducible. |
| 151 | + |
| 152 | +## Dependencies |
| 153 | + |
| 154 | +Add to `[project.dependencies]`: |
| 155 | + |
| 156 | +``` |
| 157 | +pyrage>=1.2 |
| 158 | +``` |
| 159 | + |
| 160 | +`gzip` is stdlib. No other new deps. |
| 161 | + |
| 162 | +## Docs |
| 163 | + |
| 164 | +CLAUDE.md: |
| 165 | +- Add `set-encryption-passphrase` to the commands list. |
| 166 | +- New gotcha: "Snapshot is mandatorily age-encrypted with a keychain- |
| 167 | + stored passphrase. `snapshot.read()` and `snapshot.write()` both |
| 168 | + require it; missing passphrase means `poll` refuses to run. Consumers |
| 169 | + must decrypt with the same passphrase." |
| 170 | +- Update the architecture bullet for `snapshot.py` to mention the |
| 171 | + gzip + age pipeline. |
| 172 | + |
| 173 | +README.md: |
| 174 | +- Add `set-encryption-passphrase` to the quick-start, before `poll`. |
| 175 | +- Update the `## Schema` section to note the file is age-encrypted JSON |
| 176 | + (gzipped before encryption). |
| 177 | +- Update the snapshot-size note: gzip brings the 350 MB worst case down |
| 178 | + to ~35 MB on disk. |
| 179 | + |
| 180 | +Full threat-model section is **cycle 3**, not here. |
| 181 | + |
| 182 | +## Out of scope (deferred) |
| 183 | + |
| 184 | +- **Recipient-key (X25519) mode.** Passphrase covers single-user. Add in |
| 185 | + 1.x if a real recipient-mode scenario shows up. |
| 186 | +- **Per-host passphrase rotation tooling.** Manual: run |
| 187 | + `set-encryption-passphrase` again on each host. No formal rotation |
| 188 | + workflow. |
| 189 | +- **Compression-level knob.** `gzip.compress(data, compresslevel=9)` |
| 190 | + hardcoded — lower levels not exposed. The CPU cost is negligible |
| 191 | + versus the MT5 round-trip and writers care more about output size than |
| 192 | + encode speed. |
| 193 | +- **Decryption error taxonomy.** One `ValueError` for any decrypt failure |
| 194 | + ("wrong passphrase or corrupt file"). Future could distinguish |
| 195 | + authentication failure from format error, but the user response is |
| 196 | + the same either way. |
| 197 | +- **Encryption-format upgrade path.** If we ever change algorithms |
| 198 | + (recipient mode, different compressor), a new file extension is the |
| 199 | + signal. No in-band version byte. |
0 commit comments