Add secret manager (#40)

* docs: add secret manager design

* feat(secrets): add encryption dependencies

* feat(secrets): add secret name validation

* feat(secrets): add SecretManager trait and types

* feat(secrets): add AES-256-GCM-SIV encryption

* feat(secrets): add secrets tables to v1 migration

* feat(secrets): add EncryptedSecretManager implementation

* feat(secrets): add HTTP request/response models

* feat(secrets): add HTTP handlers for secrets API

* feat(secrets): register secret API routes

* feat(secrets): wire SecretManager into RivetEngine

* feat(source): refactor Source enum for secret references

- Add Credential enum (None, SecretRef) for storing credential references
- Add ResolvedCredential<'a> borrowed view for runtime credential access
- Add with_resolved_credential() closure pattern for scoped plaintext
- Update connection_string() to accept ResolvedCredential parameter
- Remove plaintext password fields from Source variants

* feat(datafetch): add secrets parameter to DataFetcher trait

- Update discover_tables() and fetch_table() signatures
- Add resolve_connection_string() helper in duckdb and postgres drivers
- Update FetchOrchestrator to store and use secret_manager

* feat(engine): add secret_key builder method

- Add secret_key field to RivetEngineBuilder
- Auto-populate from RIVETDB_SECRET_KEY env var in new()
- Allow override via .secret_key() method for tests
- Reorder build() to create secret_manager before orchestrator

* chore: add rand dev-dependency for test key generation

* test: update tests for secret manager integration

- Update datafetch tests for new API signatures
- Add test secret key generation to TestHarness
- Update PostgresFixture to use SecretRef credentials
- Store password as secret before Postgres connection tests

* fix(secrets): make secret manager mandatory with fail-fast

RIVETDB_SECRET_KEY is now required at engine startup. Without it,
connections with credentials would fail silently at query time.

* refactor(secrets): decouple metadata from encrypted storage

Split CatalogManager secret methods into metadata (all providers) and
encrypted storage (EncryptedSecretManager only). Removes FK constraint
between tables. Adds rollback on metadata failure with test coverage.

* refactor(secrets): extract provider type as constant

* refactor(source): move connection string logic to drivers

* refactor(secrets): introduce SecretBackend abstraction

Extract storage logic into SecretBackend trait, separating metadata
coordination from value persistence to enable pluggable backends.

* refactor(secrets): make SecretManager mandatory in APIs

* fix(secrets): surface provider_ref in metadata and backend

* refactor(secrets): split put into distinct create/update flows

* fix(secrets): propagate backend errors on delete

* feat(secrets): implement three-phase delete with status column

* fix(secrets): use db constraint to prevent create race

* fix(secrets): allow delete retry for pending_delete secrets

* fix(http): include validation hint in InvalidName API error

* fix(secrets): add optimistic locking for concurrent creates

* test(secrets): add PUT /secrets/{name} tests and fix flaky test

* prevent log spam; truncate invalid secret names to max characters

* update design doc

* cargo fmt

* clippy fixes

* allow for default security key to simplify onboarding

* remove duplication with error messages

* improve error message

* remove duplicate row struct

* simplify into query builder

Author: zfarrell

Cargo.lock

@@ -37,8 +37,11 @@ tracing-subscriber = { version = "0.3", features = ["env-filter"] }
 thiserror = "2.0.17"
 uuid = { version = "1.11", features = ["v4"] }
 urlencoding = "2.1"
+aes-gcm-siv = "0.11"
+base64 = "0.22"
 
 [dev-dependencies]
 testcontainers = "0.23"
 testcontainers-modules = { version = "0.11", features = ["postgres"] }
 proptest = "1.6"
+rand = "0.8"