Skip to content

feat(identity): envelope encryption for per-user secret values (#876 machinery) - #1644

Open
DmitriyG228 wants to merge 2 commits into
mainfrom
feat/secret-field-crypto
Open

DmitriyG228 wants to merge 2 commits into
mainfrom
feat/secret-field-crypto

Conversation

@DmitriyG228

@DmitriyG228 DmitriyG228 commented Sep 6, 2026

Copy link
Copy Markdown
Member

Machinery for #876, built now because the calendar-OAuth connect flow will store Google refresh
tokens
and those must not land in users.data in the clear. An ICS URL leaking exposes one
calendar's contents; a refresh token leaking is ongoing, silent read access until revoked.

COMMITMENTS IN THIS PR

None. No published terms, no dates, no rights, no spend. Additive code that nothing calls; a new
runtime dependency (cryptography) is declared and locked. Behaviour is byte-identical until a call
site opts in.

The shape

KEK  32 bytes, operator-held, VEXA_SECRETS_KEK — never in the database
  wraps
DEK  32 bytes per user, random, stored WRAPPED in users.data["_secret_dek"]
  encrypts
each value -> "enc:v1:<b64 nonce>:<b64 ciphertext>"

Rotating the KEK re-wraps N small DEKs instead of re-encrypting every secret — that is the entire
reason for the middle key, and it is tested (test_rotating_the_kek_is_a_rewrap_not_a_reencrypt).
The JSONB structure is untouched: keys stay keys, only values change shape, so nothing querying this
column learns anything new.

Three properties worth reviewing

Ciphertexts are bound to (user, field) through AES-GCM additional data. A value lifted from one
user's row into another's, or from webhook_secret into stt_token, fails authentication rather
than decrypting somewhere it does not belong. Both directions tested.

Dual-read, never dual-write. An unprefixed value is pre-migration plaintext and reads back
unchanged, so a database mid-migration keeps working. There is deliberately no matching passthrough
on encrypt — once a cipher is configured everything written is encrypted, which makes migration a
read-then-write sweep rather than a mode that can be left half-on.

A missing KEK stops the service (#876 A4). require_readable() raises when a row holds
ciphertext and no key is configured. The failure this exists to prevent is a KEK quietly absent in
one environment and a service carrying on — serving, or worse re-writing, values in the clear.
SecretCipher.from_env() also proves the cryptography import at configure time, so a KEK set into
an image without the wheel fails at boot rather than when the first user connects a calendar.

Finding: #876's proposed seam is dead code

The issue says to put this "behind the existing SecretsPort seam" in identity_core. Nothing
imports identity_core
grep -rl identity_core across core/ and services/ returns no
consumer outside its own tests, and core/identity/services/admin-api/Dockerfile copies only
admin-api's own src. Encryption placed there would never have executed in the running service.

So this lives in admin_api/app/field_crypto.py, next to main.py and calendars.py — where
users.data is actually written. SecretsPort is a real idea (audited brokering of named operator
secrets) but it is a different concern from encrypting user-owned values at rest, and it is not
currently wired to anything. Worth deciding separately whether to wire or retire it.

Dependency

cryptography>=45,<51 — Apache-2.0 OR BSD-3, FINOS Category A. Resolves to 50.0.1, the same version
already locked in core/meetings/services/mcp/uv.lock. The stdlib has no AES, so a dependency is
unavoidable for AES-256-GCM. uv.lock regenerated (adds cffi, pycparser); the Dockerfile runs
uv sync --frozen, so the lock had to carry it.

Scope — deliberately narrow

In: the cipher, its tests, the dependency.
Out: every call site. The four existing plaintext fields (stt_token, LLM api_key,
webhook_secret, ics_url) are untouched, there is no migration sweep, and no KEK is provisioned
in any environment.

This is the narrow reading of "encrypt first": build the machinery and use it for new secrets —
the OAuth tokens — from day one, then migrate the existing four on the same machinery. Full #876
stays open, and becomes "wire four call sites and run the sweep" rather than a from-scratch design.

Tests

tests/test_field_crypto.py, 19 cases, pure — no containers, no database.

19 passed

Not run: the rest of the admin-api suite. tests/conftest.py starts Postgres and Redis via
testcontainers, and container workloads do not run on the founder's laptop — that belongs on bbb
or in CI. No file in this PR is imported by any existing test.


Contribution rights

  • Independent: I created this contribution, or otherwise have the right to submit it
    under Apache-2.0, and it is not owned or controlled by an employer, client, or other entity.
  • Employer/client authorization required: an employer, client, or other entity owns or
    may control this contribution. I am requesting Vexa's private corporate-authorization process.
  • Unsure: I need a private rights review before merge.

Every commit must also carry the contributor's own DCO Signed-off-by line. Selecting the
independent path means no individual CLA is required.

Section restored by an agent because the PR body was written without the repo template. Nothing
is ticked and nothing will be
— the template says an agent may explain the choices but must not
select one, so this is left for a human. The commits also lack Signed-off-by, which is the same
kind of certification; sign them with:
git rebase --signoff origin/main && git push --force-with-lease

…machinery)

KEK (operator env, never in the DB) wraps a per-user DEK (stored wrapped in users.data), which
encrypts each secret value to enc:v1:<nonce>:<ct>. Rotating the KEK re-wraps N small DEKs instead
of re-encrypting every secret — the whole reason for the middle key. JSONB structure is untouched:
keys stay keys, only values change shape.

Every ciphertext is bound to (user_id, field) through AES-GCM additional data, so a value lifted
between users or between fields fails authentication instead of decrypting somewhere it does not
belong. Both are tested.

Dual-read, never dual-write: an unprefixed value is pre-migration plaintext and reads back
unchanged, so a database mid-migration keeps working; there is no matching passthrough on encrypt.
And #876 A4 — require_readable() refuses to continue when a row holds ciphertext and no KEK is
configured, because the failure this exists to prevent is a key going missing in one environment
and the service serving, or rewriting, secrets in the clear.

FINDING, and it moves this file: #876 proposes putting this 'behind the existing SecretsPort seam'
in identity_core. Nothing imports identity_core — grep across core/ and services/ returns no
consumer outside its own tests, and the admin-api Dockerfile copies only admin-api's own src. That
seam is dead code, so encryption placed there would never have run. This lives in admin_api/app/
next to main.py and calendars.py, which is where users.data is actually written.

Adds cryptography (Apache-2.0 OR BSD-3, Category A; 50.0.1, matching the pin already resolved in
core/meetings/services/mcp/uv.lock). The stdlib has no AES. Import is proven at configure time so
a KEK set into an image without the wheel fails at boot, not at the first user.

Scope: machinery only. No call site is wired, no migration sweep, no KEK provisioned in any
environment. Behaviour is identical until a caller opts in. [19 new tests]
uv.lock regenerated: cryptography 50.0.1 + cffi + pycparser. The Dockerfile runs uv sync --frozen,
so the lock must carry it or the image build fails.
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

🃏 Merge card — #1644

check what it needs
Value missing state: value-signed (the value sign-off)
Diff maintainer self-review — @DmitriyG228 holds the commit bit (no separate non-author review required)

Not mergeable yet — every row above must be accepted before merge (choke point 1). Fill in what's ❌ above, then this clears automatically.

How a PR reaches merge: the merge bar.

@DmitriyG228 DmitriyG228 added the docs: none PR touches a product surface but needs no docs change (D6c waiver, give reason) label Sep 6, 2026
@DmitriyG228

Copy link
Copy Markdown
Member Author

docs: none reason (D6c): machinery only — no call site is wired, no environment has a KEK, and behaviour is byte-identical until a caller opts in. The operator-facing documentation this will need (VEXA_SECRETS_KEK, how to generate one, the rotation procedure, and the migration sweep) belongs with the first PR that actually reads a secret through the cipher; writing it now would document a configuration that does nothing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs: none PR touches a product surface but needs no docs change (D6c waiver, give reason)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant