|
| 1 | +# Claude Code Context for Client Compatibility Tests |
| 2 | + |
| 3 | +## What This Is |
| 4 | + |
| 5 | +Docker Compose-based test framework that runs real PostgreSQL client libraries against duckgres and collects results into a DuckDB database. Lives under `scripts/client-compat/`. |
| 6 | + |
| 7 | +## Directory Layout |
| 8 | + |
| 9 | +``` |
| 10 | +scripts/client-compat/ |
| 11 | +├── CLAUDE.md # you are here |
| 12 | +├── README.md # human-facing docs |
| 13 | +├── docker-compose.yml # all services |
| 14 | +├── justfile # task runner |
| 15 | +├── Dockerfile.results # results-gatherer image |
| 16 | +├── results_gatherer.py # HTTP server that collects results into DuckDB |
| 17 | +├── report_client.py # Python helper for reporting (shared by Python clients) |
| 18 | +├── entrypoint.sh # log-tee wrapper (shared by all clients) |
| 19 | +├── queries.yaml # shared query catalog executed by every client |
| 20 | +├── clients/ # one directory per client |
| 21 | +│ ├── psycopg/ |
| 22 | +│ ├── pgx/ |
| 23 | +│ ├── psql/ |
| 24 | +│ ├── jdbc/ |
| 25 | +│ ├── tokio-postgres/ |
| 26 | +│ ├── node-postgres/ |
| 27 | +│ ├── sqlalchemy/ |
| 28 | +│ └── harlequin/ |
| 29 | +└── results/ # output volume (.gitignored) |
| 30 | +``` |
| 31 | + |
| 32 | +## Lifecycle / How It Works |
| 33 | + |
| 34 | +1. `just all` → `docker compose up -d` starts duckgres, results-gatherer, and all clients |
| 35 | +2. Clients block on healthchecks (`duckgres` TCP + `results-gatherer` HTTP `/health`) |
| 36 | +3. Each client runs tests, POSTing `{client, suite, test_name, status, detail}` to `POST /result` |
| 37 | +4. The justfile runs `docker wait` on all client containers (handles crashes gracefully) |
| 38 | +5. Once all clients exit, the justfile `docker exec`s a `POST /shutdown` to the gatherer |
| 39 | +6. Gatherer prints report, exports `results_<ts>.duckdb` and `.json`, exits 0 |
| 40 | +7. `docker compose down -v` tears everything down |
| 41 | + |
| 42 | +**Key design point**: the gatherer does NOT track which clients are done. The justfile handles all lifecycle coordination via `docker wait`. This avoids hangs when clients crash before reporting. |
| 43 | + |
| 44 | +## Reporting Protocol |
| 45 | + |
| 46 | +Clients report results via HTTP to `$RESULTS_URL` (defaults to `http://results-gatherer:8080`): |
| 47 | + |
| 48 | +``` |
| 49 | +POST /result |
| 50 | +{ |
| 51 | + "client": "psycopg", # must match the client name |
| 52 | + "suite": "catalog_views", # test grouping |
| 53 | + "test_name": "pg_database", # unique within suite |
| 54 | + "status": "pass", # "pass" or "fail" |
| 55 | + "detail": "4 rows" # optional context (row count, error message) |
| 56 | +} |
| 57 | +``` |
| 58 | + |
| 59 | +All reporting is fire-and-forget (errors silently swallowed) so clients also work standalone without the gatherer. |
| 60 | + |
| 61 | +### Python clients |
| 62 | + |
| 63 | +Use `report_client.py` (copied into the image): |
| 64 | + |
| 65 | +```python |
| 66 | +from report_client import ReportClient |
| 67 | + |
| 68 | +rc = ReportClient("my_client") |
| 69 | +rc.report("suite_name", "test_name", "pass", "detail") |
| 70 | +``` |
| 71 | + |
| 72 | +### Non-Python clients |
| 73 | + |
| 74 | +Implement HTTP POST directly. See `clients/pgx/main.go` (Go), `clients/jdbc/.../JdbcCompatTest.java` (Java), `clients/tokio-postgres/src/main.rs` (Rust), or `clients/psql/test_psql.sh` (curl) for examples. |
| 75 | + |
| 76 | +## Adding a New Client |
| 77 | + |
| 78 | +Name the directory after the **client library**, not the language (e.g. `tokio-postgres` not `rust`, `pgx` not `go`). |
| 79 | + |
| 80 | +### 1. Create the client directory |
| 81 | + |
| 82 | +``` |
| 83 | +clients/<name>/ |
| 84 | +├── Dockerfile |
| 85 | +└── <test script> |
| 86 | +``` |
| 87 | + |
| 88 | +### 2. Write the Dockerfile |
| 89 | + |
| 90 | +Build context is `scripts/client-compat/` (not the client dir). COPY paths must be relative to that: |
| 91 | + |
| 92 | +```dockerfile |
| 93 | +FROM python:3.12-slim-bookworm |
| 94 | + |
| 95 | +RUN pip install --no-cache-dir <deps> pyyaml |
| 96 | + |
| 97 | +COPY entrypoint.sh /entrypoint.sh |
| 98 | +COPY queries.yaml /queries.yaml |
| 99 | +COPY report_client.py /report_client.py # Python clients only |
| 100 | +COPY clients/<name>/test_<name>.py /test_<name>.py |
| 101 | + |
| 102 | +ENTRYPOINT ["/entrypoint.sh", "python", "/test_<name>.py"] |
| 103 | +``` |
| 104 | + |
| 105 | +For compiled languages, use a multi-stage build (see `clients/pgx/Dockerfile` or `clients/tokio-postgres/Dockerfile`). |
| 106 | + |
| 107 | +### 3. Implement the test script |
| 108 | + |
| 109 | +Every client must: |
| 110 | + |
| 111 | +1. **Wait for duckgres** — poll with `SELECT 1`, retry up to 30 times with 1s sleep |
| 112 | +2. **Run shared queries** — load `/queries.yaml`, execute each entry, report pass/fail |
| 113 | +3. **Run client-specific tests** — whatever the library supports (DDL/DML, prepared statements, COPY, ORM, etc.) |
| 114 | +4. **Connect with TLS** — use `sslmode=require` and accept self-signed certs |
| 115 | +5. **Exit non-zero on failure** — so `docker wait` captures it |
| 116 | + |
| 117 | +The shared queries YAML format: |
| 118 | +```yaml |
| 119 | +- suite: catalog_views |
| 120 | + name: pg_database |
| 121 | + sql: SELECT datname FROM pg_database WHERE datallowconn |
| 122 | +``` |
| 123 | +
|
| 124 | +### 4. Register in docker-compose.yml |
| 125 | +
|
| 126 | +Add a service block (copy an existing one and change names/paths): |
| 127 | +
|
| 128 | +```yaml |
| 129 | + <name>: |
| 130 | + build: |
| 131 | + context: . |
| 132 | + dockerfile: clients/<name>/Dockerfile |
| 133 | + container_name: compat-<name> |
| 134 | + depends_on: |
| 135 | + duckgres: |
| 136 | + condition: service_healthy |
| 137 | + results-gatherer: |
| 138 | + condition: service_healthy |
| 139 | + volumes: |
| 140 | + - ./results:/results |
| 141 | + environment: |
| 142 | + PGHOST: duckgres |
| 143 | + PGPORT: "5432" |
| 144 | + PGUSER: postgres |
| 145 | + PGPASSWORD: postgres |
| 146 | + RESULTS_URL: http://results-gatherer:8080 |
| 147 | + LOG_DIR: /results/<name> |
| 148 | +``` |
| 149 | +
|
| 150 | +### 5. Register in justfile |
| 151 | +
|
| 152 | +Two changes: |
| 153 | +
|
| 154 | +1. Add the container name to the `clients` variable: |
| 155 | + ``` |
| 156 | + clients := "compat-psycopg compat-pgx ... compat-<name>" |
| 157 | + ``` |
| 158 | +
|
| 159 | +2. Add a standalone target: |
| 160 | + ``` |
| 161 | + # Run <name> compatibility tests (standalone, no report) |
| 162 | + [group('client')] |
| 163 | + <name>: build |
| 164 | + docker compose run --rm <name> |
| 165 | + docker compose down -v |
| 166 | + ``` |
| 167 | +
|
| 168 | +### 6. Update README.md |
| 169 | +
|
| 170 | +Add the client to the tables in the Clients, Client-Specific Suites, and available targets sections. |
| 171 | +
|
| 172 | +## Common Patterns |
| 173 | +
|
| 174 | +### Connection parameters |
| 175 | +
|
| 176 | +Always use env vars: `PGHOST`, `PGPORT`, `PGUSER`, `PGPASSWORD`. Defaults should match docker-compose values (`duckgres`, `5432`, `postgres`, `postgres`). |
| 177 | +
|
| 178 | +### TLS |
| 179 | +
|
| 180 | +Duckgres auto-generates a self-signed cert. Clients must accept it: |
| 181 | +- Python (psycopg2): `sslmode="require"` (no cert validation by default) |
| 182 | +- Go (pgx): `InsecureSkipVerify: true` in tls.Config |
| 183 | +- Java (JDBC): `sslmode=require&sslfactory=org.postgresql.ssl.NonValidatingFactory` |
| 184 | +- Rust (tokio-postgres): `danger_accept_invalid_certs(true)` on TlsConnector |
| 185 | +
|
| 186 | +### Suites |
| 187 | +
|
| 188 | +Group related tests under a suite name. Common conventions: |
| 189 | +- `connection` — TLS, auth, server version, driver info |
| 190 | +- `ddl_dml` / `core_ddl_dml` — CREATE, INSERT, UPDATE, DELETE, DROP |
| 191 | +- Library-specific features get their own suite (`batch`, `copy`, `orm`, `prepared`, etc.) |
| 192 | +
|
| 193 | +## Results Database |
| 194 | +
|
| 195 | +Exported to `results/results_<timestamp>.duckdb` with: |
| 196 | +
|
| 197 | +- `queries` table — the shared query catalog from `queries.yaml` |
| 198 | +- `results` table — all test outcomes (client, suite, test_name, status, detail, ts) |
| 199 | +- `coverage` view — LEFT JOIN of queries to results (shows gaps) |
| 200 | +
|
| 201 | +Open with `just query-last-run` or `duckdb results/results_*.duckdb`. |
0 commit comments