Skip to content

Add postgres configuration store for on-demand schema loading#179

Open
manasag wants to merge 5 commits intomainfrom
feature/postgres-configuration-store
Open

Add postgres configuration store for on-demand schema loading#179
manasag wants to merge 5 commits intomainfrom
feature/postgres-configuration-store

Conversation

@manasag
Copy link

@manasag manasag commented Mar 11, 2026

Summary

  • Adds alternate configuration mode where the connector reads collection schemas from a PostgreSQL config store (config_tables.raw_schema) instead of filesystem JSON files
  • Configuration is fetched per-request rather than cached at startup, enabling dynamic schema management without restarts
  • New configuration-store crate with ConfigurationMode resolver and PostgresConfigurationStore backed by deadpool-postgres connection pool
  • TLS support via native-tls for postgres connections (?sslmode=require in URL)
  • Credential-safe Debug impls (postgres URL redacted in logs)
  • Bumps Rust toolchain from 1.83.0 to 1.85.0 to fix CI build failure (wit-bindgen 0.51.0 requires edition2024)
  • Updates nixpkgs and crane flake inputs to fix cargo-audit failure parsing CVSS 4.0 scores in newer RustSec advisories

Env vars

Variable Purpose
HASURA_CONFIGURATION_MODE json (default) or a postgres connection URL
HASURA_CONFIGURATION_CONNECTOR_ID Required in postgres mode — scopes config queries
HASURA_CONFIGURATION_SCHEMA Postgres schema name (default: connector_config)

Behavior in postgres mode

  • /query: fetches schemas for all collections referenced in the request (primary + relationship targets) from config_tables.raw_schema, merges them into a single configuration
  • /schema: returns empty response (schema managed externally)
  • /mutation: returns 501 Not Implemented (native mutations not in postgres store)
  • Connection URI: read from config_metadata table, falls back to MONGODB_DATABASE_URI env var

Changes in this update

  • Relationship support: queries involving relationships (e.g. AlbumTrackGenre) now work correctly. The connector extracts all collection names from the QueryRequest (primary collection + all collection_relationships targets) and fetches/merges their schemas from the config store.
  • Schema fix: config_tables query uses is_deleted = false instead of deleted_at IS NULL to match the actual table schema.

Test plan

  • All 145 existing unit tests pass
  • All 27 connector-direct integration tests pass in postgres mode (grouping, filtering, nested collections, relationships, remote relationships, expressions)
  • Relationship queries (joins, aggregates over related collections, groups through relationships) verified working
  • Manual end-to-end test with local PostgreSQL + MongoDB containers
  • Engine-dependent integration tests (require v3-engine, not tested locally)

🤖 Generated with Claude Code

manasag and others added 3 commits March 10, 2026 20:04
Adds an alternate configuration mode where the connector reads collection
schemas from a PostgreSQL config store (config_tables.raw_schema) instead
of filesystem JSON files. Configuration is fetched per-request rather than
cached at startup, enabling dynamic schema management.

Key changes:
- New `configuration-store` crate with ConfigurationMode resolver and
  PostgresConfigurationStore backed by deadpool-postgres connection pool
- ConnectorConfig enum (Static/Postgres) replaces MongoConfiguration as
  the SDK Configuration type
- Per-query: fetches only the requested collection's schema from postgres
- get_schema returns empty in postgres mode (schema managed externally)
- Mutations unsupported in postgres mode (native mutations not stored)
- TLS support via native-tls for postgres connections with sslmode
- Connection URI read from config_metadata or MONGODB_DATABASE_URI env var

Env vars:
- HASURA_CONFIGURATION_MODE: "json" (default) or postgres URL
- HASURA_CONFIGURATION_CONNECTOR_ID: required for postgres mode
- HASURA_CONFIGURATION_SCHEMA: postgres schema name (default: connector_config)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
getrandom 0.3.4 (via rand_core 0.9.5) pulls in wit-bindgen 0.51.0
which requires edition2024, stabilized in Rust 1.85.0. This fixes
the Nix build failure in CI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The cargo-audit step fails because the RustSec advisory database now
includes advisories with CVSS 4.0 scores, which older cargo-audit
versions cannot parse. Updating nixpkgs brings in a newer cargo-audit
that supports CVSS 4.0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@manasag manasag force-pushed the feature/postgres-configuration-store branch 5 times, most recently from 91aa883 to d768914 Compare March 12, 2026 03:05
- Add read_collections_configuration() to fetch and merge schemas for
  multiple collections in a single request. Previously only the primary
  collection's schema was loaded, causing "Unknown collection" errors
  when queries referenced related collections (e.g. Album -> Track).
- Extract all collection names from QueryRequest (primary + relationship
  targets) before resolving configuration.
- Fix config_tables query to use is_deleted = false instead of
  deleted_at IS NULL to match the actual table schema.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@manasag manasag force-pushed the feature/postgres-configuration-store branch from d768914 to 0a58c2b Compare March 12, 2026 04:04
tokio-postgres 0.7.13+ pulls in rand 0.9 → getrandom 0.3 → wasip2 →
wit-bindgen 0.51.0 which requires Rust edition 2024 (Cargo 1.85+).
Our Nix build uses Rust 1.83.0, so pin to <0.7.13 to keep CI working.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant