Skip to content

Commit 173c25d

Browse files
authored
Add dbt to runner test suite, other improvements. (#265)
* Add initial dbt test. * Fix some runner issues, rename some queries to avoi confusion. * Add stub flag for queries that return something, rather than nothing at all. * Bump jdbc to avoid security warning.
1 parent e3c9e38 commit 173c25d

File tree

16 files changed

+471
-32
lines changed

16 files changed

+471
-32
lines changed

scripts/client-compat/CLAUDE.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ scripts/client-compat/
2525
│ ├── tokio-postgres/
2626
│ ├── node-postgres/
2727
│ ├── sqlalchemy/
28+
│ ├── dbt/
2829
│ └── harlequin/
2930
└── results/ # output volume (.gitignored)
3031
```
@@ -118,6 +119,7 @@ The shared queries YAML format:
118119
```yaml
119120
- suite: catalog_views
120121
name: pg_database
122+
stub: true # optional — flags hardcoded dummy return values
121123
sql: SELECT datname FROM pg_database WHERE datallowconn
122124
```
123125
@@ -194,8 +196,8 @@ Group related tests under a suite name. Common conventions:
194196
195197
Exported to `results/results_<timestamp>.duckdb` with:
196198
197-
- `queries` table — the shared query catalog from `queries.yaml`
199+
- `queries` table — the shared query catalog from `queries.yaml` (includes `stub` boolean)
198200
- `results` table — all test outcomes (client, suite, test_name, status, detail, ts)
199-
- `coverage` view — LEFT JOIN of queries to results (shows gaps)
201+
- `coverage` view — LEFT JOIN of queries to results (shows gaps, includes `stub` flag)
200202
201203
Open with `just query-last-run` or `duckdb results/results_*.duckdb`.

scripts/client-compat/README.md

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ just query-last-run
3434
| `just view-last-run` | Print the full results table |
3535
| `just query-last-run` | Open the latest results database in DuckDB CLI |
3636

37-
Available single-client targets: `psycopg`, `pgx`, `psql`, `jdbc`, `tokio-postgres`, `node-postgres`, `sqlalchemy`.
37+
Available single-client targets: `psycopg`, `pgx`, `psql`, `jdbc`, `tokio-postgres`, `node-postgres`, `sqlalchemy`, `dbt`.
3838

3939
## Clients
4040

@@ -47,6 +47,7 @@ Available single-client targets: `psycopg`, `pgx`, `psql`, `jdbc`, `tokio-postgr
4747
| tokio-postgres | Rust 1.84 | tokio-postgres 0.7 | rust:1.84-bookworm |
4848
| node-postgres | Node.js 22 | pg 8.x | node:22-bookworm-slim |
4949
| sqlalchemy | Python 3.12 | SQLAlchemy 2.x + psycopg2 | python:3.12-slim |
50+
| dbt | Python 3.12 | dbt-postgres (psycopg2) | python:3.12-slim |
5051

5152
## Architecture
5253

@@ -91,15 +92,17 @@ Every client runs two categories of tests:
9192

9293
### Shared Queries (`queries.yaml`)
9394

94-
A YAML catalog of 44 queries across 5 suites that every client executes. A query passes if it executes without error (row counts are reported but not validated).
95+
A YAML catalog of 58 queries across 5 suites that every client executes. A query passes if it executes without error (row counts are reported but not validated).
96+
97+
Queries may be tagged `stub: true` to indicate the underlying function or view returns a hardcoded dummy value (empty string, NULL, 0, false, or empty result set) rather than meaningful data. This distinguishes "won't crash your client" from "returns useful information" in the results.
9598

9699
| Suite | Count | Purpose |
97100
|-------|-------|---------|
98101
| `catalog_views` | 7 | Core pg_catalog views (`pg_database`, `pg_namespace`, `pg_type`, etc.) |
99-
| `catalog_funcs` | 14 | PostgreSQL functions (`format_type`, `version()`, `current_setting`, etc.) |
102+
| `catalog_funcs` | 26 | PostgreSQL functions (`format_type`, `version()`, `pg_get_indexdef`, size functions, etc.) |
100103
| `info_schema` | 4 | `information_schema.tables`, `.columns`, `.schemata` |
101104
| `catalog_joins` | 5 | Multi-table joins that real tools emit (psql `\dt`, `\dn`, `\l` queries) |
102-
| `catalog_stubs` | 14 | Stub views that must exist but return empty (`pg_matviews`, `pg_policy`, etc.) |
105+
| `catalog_stubs` | 16 | Stub views that must exist but return empty (`pg_matviews`, `pg_policy`, etc.) |
103106

104107
### Client-Specific Suites
105108

@@ -114,6 +117,7 @@ Each client tests driver-specific features beyond the shared catalog:
114117
| tokio-postgres | `connection`, `ddl_dml`, `prepared` |
115118
| node-postgres | `connection`, `ddl_dml`, `prepared`, `result_metadata` |
116119
| sqlalchemy | `connection`, `core_ddl_dml`, `orm`, `inspection`, `raw_params` |
120+
| dbt | `dbt_lifecycle` (`dbt debug`, `dbt run`, `dbt test`, `dbt docs generate`) |
117121

118122
## Results
119123

@@ -139,7 +143,8 @@ results/
139143
CREATE TABLE queries (
140144
suite VARCHAR,
141145
name VARCHAR,
142-
sql VARCHAR
146+
sql VARCHAR,
147+
stub BOOLEAN DEFAULT false -- true = hardcoded dummy return value
143148
);
144149

145150
-- Test outcomes
@@ -154,7 +159,7 @@ CREATE TABLE results (
154159

155160
-- Coverage: LEFT JOIN of queries to results (shows untested queries)
156161
CREATE VIEW coverage AS
157-
SELECT q.suite, q.name, q.sql,
162+
SELECT q.suite, q.name, q.sql, q.stub,
158163
r.client, r.status, r.detail, r.ts
159164
FROM queries q
160165
LEFT JOIN results r ON q.suite = r.suite AND q.name = r.test_name;
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
FROM python:3.12-slim-bookworm
2+
3+
RUN apt-get update && apt-get install -y --no-install-recommends git && rm -rf /var/lib/apt/lists/*
4+
RUN pip install --no-cache-dir dbt-postgres pyyaml
5+
6+
COPY entrypoint.sh /entrypoint.sh
7+
COPY queries.yaml /queries.yaml
8+
COPY report_client.py /report_client.py
9+
COPY clients/dbt/test_dbt.py /test_dbt.py
10+
COPY clients/dbt/project /dbt_project
11+
12+
ENTRYPOINT ["/entrypoint.sh", "python", "/test_dbt.py"]
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
name: duckgres_compat
2+
version: "1.0.0"
3+
4+
profile: duckgres
5+
6+
model-paths: ["models"]
7+
test-paths: ["tests"]
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
-- A table materialization that aggregates from the view
2+
{{ config(materialized='table') }}
3+
4+
SELECT
5+
count(*) AS total_rows,
6+
min(value) AS min_value,
7+
max(value) AS max_value,
8+
avg(value) AS avg_value
9+
FROM {{ ref('staging_numbers') }}
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
version: 2
2+
3+
models:
4+
- name: staging_numbers
5+
columns:
6+
- name: id
7+
tests:
8+
- not_null
9+
- unique
10+
11+
- name: agg_summary
12+
columns:
13+
- name: total_rows
14+
tests:
15+
- not_null
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
-- A simple view model that generates test data
2+
{{ config(materialized='view') }}
3+
4+
SELECT
5+
i AS id,
6+
'item_' || i::VARCHAR AS name,
7+
(i * 10.5)::DOUBLE AS value
8+
FROM generate_series(1, 100) AS t(i)
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
duckgres:
2+
target: dev
3+
outputs:
4+
dev:
5+
type: postgres
6+
host: "{{ env_var('PGHOST', 'duckgres') }}"
7+
port: "{{ env_var('PGPORT', '5432') | int }}"
8+
user: "{{ env_var('PGUSER', 'postgres') }}"
9+
pass: "{{ env_var('PGPASSWORD', 'postgres') }}"
10+
dbname: postgres
11+
schema: public
12+
sslmode: require

0 commit comments

Comments
 (0)