Skip to content

Commit b95f829

Browse files
Merge pull request #2914 from maziyarpanahi/feature/issue-919-key-lifecycle
feat: add versioned key lifecycle management
2 parents bb32899 + 88bfa33 commit b95f829

7 files changed

Lines changed: 612 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
### Added
1111

12+
- Added an offline, versioned key-lifecycle helper and operator guide for
13+
audit-key rotation, retired-key verification, surrogate-vault re-keying,
14+
environment isolation, and file-permission hygiene without serializing keys.
1215
- Added conservative two- and three-column PDF reading-order reconstruction,
1316
preserving source word bboxes and character-span projection while leaving
1417
single-column extraction byte-for-byte compatible with the source-order path.

docs/brand/system/publication.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,7 @@ classification:
204204
- pii-smart-merging.md
205205
- security/no-raw-phi-logging.md
206206
- security/tamper-evident-audit-log.md
207+
- security/key-management.md
207208
- integrations-langchain.md
208209
- integrations/arrow-flight.md
209210
- integrations-haystack.md

docs/security/key-management.md

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
# Key management and rotation
2+
3+
OpenMed keeps surrogate and audit-key custody local to the caller. It does not
4+
send keys to a service, persist them in reports, or provide a cloud escrow. This
5+
guide describes the operator-owned lifecycle for generating, storing, rotating,
6+
retiring, and recovering keys without placing raw PHI or key material in logs.
7+
8+
## Separate keys by purpose
9+
10+
Use independent keys for each environment and purpose:
11+
12+
- audit-report HMAC signing;
13+
- cross-document surrogate vaults;
14+
- patient-consistent date shifting; and
15+
- any application-specific stable content hashes.
16+
17+
Do not reuse a production key in development, tests, or multiple tenants. Key
18+
identifiers are metadata and may appear in audit reports, so keep them short,
19+
stable, and PHI-free.
20+
21+
## Generate and rotate audit keys
22+
23+
`KeyLifecycle` manages one active key and retained verification keys entirely in
24+
memory. New keys are generated with Python's `secrets` module unless the caller
25+
supplies key material from its own secret store.
26+
27+
```python
28+
from openmed.core import KeyLifecycle
29+
30+
keys = KeyLifecycle.generate(prefix="audit")
31+
first_id = keys.active_key_id
32+
signed_report = keys.sign_audit(report)
33+
34+
rotation = keys.rotate()
35+
assert rotation.key_id != first_id
36+
37+
# The old key is retired for signing but retained for verification.
38+
assert keys.verify_audit(signed_report)
39+
```
40+
41+
After a restart, load the caller-owned active and retained keys explicitly:
42+
43+
```python
44+
import os
45+
46+
from openmed.core import KeyLifecycle
47+
48+
keys = KeyLifecycle.from_keys(
49+
{
50+
"audit-v0001": os.environ["OPENMED_AUDIT_KEY_V1"],
51+
"audit-v0002": os.environ["OPENMED_AUDIT_KEY_V2"],
52+
},
53+
active_key_id="audit-v0002",
54+
prefix="audit",
55+
)
56+
```
57+
58+
Keep a retired audit key for at least as long as its signed evidence must remain
59+
verifiable. Removing it makes those reports cryptographically unverifiable.
60+
`metadata()` is safe to inventory because it returns only IDs, versions, and
61+
active/retired states; it never returns raw keys.
62+
63+
## Rotate a surrogate vault epoch
64+
65+
`SurrogateVault` derives versioned epoch IDs from its caller-supplied root
66+
secret. Rotating an epoch re-HMACs and re-encrypts the vault entries. Because a
67+
persisted vault contains no raw source surfaces, a non-empty vault requires the
68+
operator to supply the source catalog during migration:
69+
70+
```python
71+
from openmed.core import SurrogateSource, SurrogateVault
72+
73+
vault = SurrogateVault.from_file(
74+
"surrogate-vault.json",
75+
hmac_secret=os.environ["OPENMED_SURROGATE_KEY"],
76+
)
77+
sources = [SurrogateSource("synthetic-person", "NAME", "en")]
78+
result = vault.rotate(sources, revoke_previous=True)
79+
assert result.consistency is not None and result.consistency.passed
80+
```
81+
82+
The source catalog is consumed in memory and is not serialized. Epoch rotation
83+
changes the vault's derived HMAC linkage and encryption keys but does **not**
84+
replace the caller-supplied root secret. Always retain a protected backup and
85+
require a passing consistency report before committing the migration.
86+
87+
## Replace a surrogate-vault root secret
88+
89+
A lost root secret cannot be recovered from the vault file. If the root secret
90+
must change, create a separate vault with the new secret and copy mappings using
91+
the trusted in-memory source catalog. Do not overwrite the old vault in place:
92+
93+
```python
94+
import os
95+
96+
from openmed.core import KeyLifecycle, SurrogateSource, SurrogateVault
97+
98+
keys = KeyLifecycle(
99+
os.environ["OPENMED_SURROGATE_KEY_V1"],
100+
prefix="surrogate",
101+
)
102+
old_vault = SurrogateVault.from_file("vault.json", hmac_secret=keys.active_key)
103+
source = SurrogateSource("synthetic-person", "NAME", "en")
104+
surrogate = old_vault.get(
105+
source.source_text,
106+
label=source.label,
107+
lang=source.lang,
108+
)
109+
if surrogate is None:
110+
raise RuntimeError("source catalog does not match the existing vault")
111+
112+
keys.rotate(os.environ["OPENMED_SURROGATE_KEY_V2"])
113+
new_vault = SurrogateVault.from_file(
114+
"vault.next.json",
115+
hmac_secret=keys.active_key,
116+
)
117+
new_vault.get_or_create(
118+
source.source_text,
119+
label=source.label,
120+
lang=source.lang,
121+
create_surrogate=lambda _attempt: surrogate,
122+
)
123+
assert new_vault.get(
124+
source.source_text,
125+
label=source.label,
126+
lang=source.lang,
127+
) == surrogate
128+
```
129+
130+
Repeat the copy and equality check for every catalog entry, then atomically
131+
switch consumers to the new vault and root key. Replacing a vault without this
132+
mapping migration changes stable cross-document pseudonyms and can break
133+
longitudinal joins.
134+
135+
## Rotation procedure
136+
137+
1. Inventory the active PHI-free key ID and every artifact or vault that uses
138+
it; never inventory raw key bytes.
139+
2. Generate at least 32 random bytes in the deployment's approved local secret
140+
store and give the new version a non-PHI ID.
141+
3. Back up encrypted vaults and retained verification keys before migration.
142+
4. Rotate audit signing first, then verify reports made with both old and new
143+
key IDs.
144+
5. For an epoch rotation, rotate each surrogate vault with its in-memory source
145+
catalog and require a passing consistency report. For root-secret rotation,
146+
build and verify a separate migrated vault as described above.
147+
6. Deploy the new active key to every writer before retiring the old key.
148+
7. Keep old audit keys read-only for the evidence-retention period. Revoke a
149+
compromised surrogate epoch only after its entries have migrated.
150+
8. Record only dates, key IDs, owners, and verification results in the change
151+
record. Never record secrets or source identifiers.
152+
153+
Rotate on a documented cadence appropriate to the deployment and immediately
154+
after suspected exposure, operator departure, backup compromise, or accidental
155+
secret disclosure. Emergency rotation should use the same verification steps;
156+
urgency is not a reason to skip consistency checks.
157+
158+
## Environment and file-permission checklist
159+
160+
- Prefer an OS keychain, mounted secret file, or local secret manager over
161+
command-line arguments or shell history.
162+
- If environment variables are required, scope them to the service process,
163+
prevent debug dumps, and remember that child processes inherit them.
164+
- Create secret and vault files under `umask 077`; require mode `0600` for files
165+
and `0700` for their parent directory.
166+
- Keep keys, `.env` files, vault backups, and recovery bundles out of Git,
167+
container images, notebooks, fixtures, logs, crash reports, and support
168+
archives.
169+
- Restrict backup access separately from application runtime access and test
170+
restoration on synthetic data.
171+
- Never place raw PHI in a key ID, filename, exception, rotation record, or
172+
audit note.
173+
- Verify both old and new audit signatures before deleting or disabling any
174+
retained key.
175+
176+
OpenMed cannot enforce storage permissions in a caller-owned secret store. The
177+
deployment operator remains responsible for access control, backup protection,
178+
retention, destruction, and incident response.

mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,7 @@ nav:
132132
- Smart Entity Merging: pii-smart-merging.md
133133
- No-Raw-PHI Logging: security/no-raw-phi-logging.md
134134
- Tamper-evident Audit Log: security/tamper-evident-audit-log.md
135+
- Key Management and Rotation: security/key-management.md
135136
- LangChain Redaction Wrapper: integrations-langchain.md
136137
- Arrow Flight De-identification: integrations/arrow-flight.md
137138
- Haystack Redaction Component: integrations-haystack.md

openmed/core/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@
4949
detect_name_script,
5050
indic_names_match,
5151
)
52+
from .key_lifecycle import KeyLifecycle, KeyMetadata
5253
from .language_pack import (
5354
LANGUAGE_PACK_REGISTRY,
5455
LanguagePack,
@@ -180,6 +181,8 @@
180181
"AuditSignature",
181182
"AuditSpan",
182183
"DetectorInfo",
184+
"KeyLifecycle",
185+
"KeyMetadata",
183186
"AuditChain",
184187
"AuditChainEntry",
185188
"AuditChainSpan",

0 commit comments

Comments
 (0)