Skip to content

Commit 22d2290

Browse files
fuziontechfuziontech
authored
Merge pull request #41 from PostHog/feature/integration-test-suite
Add comprehensive PostgreSQL compatibility integration test suite
2 parents 60e3180 + dcfa600 commit 22d2290

19 files changed

+7636
-0
lines changed

‎INTEGRATION_TEST_PLAN.md‎

Lines changed: 1666 additions & 0 deletions
Large diffs are not rendered by default.

‎server/conn.go‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ type preparedStmt struct {
2626
isIgnoredSet bool // True if this is an ignored SET parameter
2727
isNoOp bool // True if this is a no-op command (CREATE INDEX, etc.)
2828
noOpTag string // Command tag for no-op commands
29+
described bool // True if Describe(S) was called on this statement
2930
}
3031

3132
type portal struct {
@@ -1032,6 +1033,16 @@ func formatValue(v interface{}) string {
10321033
return "t"
10331034
}
10341035
return "f"
1036+
case time.Time:
1037+
// PostgreSQL timestamp format without timezone suffix
1038+
if val.IsZero() {
1039+
return ""
1040+
}
1041+
// Use microsecond precision if there are sub-second components
1042+
if val.Nanosecond() != 0 {
1043+
return val.Format("2006-01-02 15:04:05.999999")
1044+
}
1045+
return val.Format("2006-01-02 15:04:05")
10351046
default:
10361047
// For other types, try to convert to string
10371048
return fmt.Sprintf("%v", val)
@@ -1210,6 +1221,7 @@ func (c *clientConn) handleBind(body []byte) {
12101221
stmt: ps,
12111222
paramValues: paramValues,
12121223
resultFormats: resultFormats,
1224+
described: ps.described, // Inherit from statement if Describe(S) was called
12131225
}
12141226

12151227
writeBindComplete(c.writer)
@@ -1293,6 +1305,7 @@ func (c *clientConn) handleDescribe(body []byte) {
12931305

12941306
log.Printf("[%s] Describe statement: sending RowDescription with %d columns", c.username, len(cols))
12951307
c.sendRowDescription(cols, colTypes)
1308+
ps.described = true
12961309

12971310
case 'P':
12981311
// Describe portal

‎tests/integration/README.md‎

Lines changed: 254 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
# Duckgres PostgreSQL Compatibility Integration Tests
2+
3+
This directory contains a comprehensive integration test suite to verify Duckgres compatibility with PostgreSQL 16 for OLAP workloads.
4+
5+
## Overview
6+
7+
The test suite runs queries against both a real PostgreSQL 16 instance and Duckgres, comparing results to ensure semantic equivalence. This approach catches compatibility issues that unit tests might miss.
8+
9+
### Test Architecture
10+
11+
```
12+
┌─────────────────────────────────────────────────────────────────────────┐
13+
│ Test Runner (Go) │
14+
├─────────────────────────────────────────────────────────────────────────┤
15+
│ │
16+
│ ┌─────────────────────┐ ┌─────────────────────┐ │
17+
│ │ PostgreSQL 16 │ │ Duckgres │ │
18+
│ │ (Docker) │ │ (In-Process) │ │
19+
│ │ Port: 35432 │ │ Port: dynamic │ │
20+
│ └─────────────────────┘ └─────────────────────┘ │
21+
│ │ │ │
22+
│ └────────────┬───────────────────────┘ │
23+
│ │ │
24+
│ ▼ │
25+
│ ┌──────────────────────┐ │
26+
│ │ Result Comparator │ │
27+
│ │ - Column names │ │
28+
│ │ - Row data │ │
29+
│ │ - Type coercion │ │
30+
│ └──────────────────────┘ │
31+
│ │
32+
└─────────────────────────────────────────────────────────────────────────┘
33+
```
34+
35+
## Quick Start
36+
37+
### Prerequisites
38+
39+
- Go 1.21+
40+
- Docker and Docker Compose
41+
- GCC C++ compiler (for DuckDB native bindings - provides libstdc++ for linking)
42+
43+
```bash
44+
# Fedora/RHEL
45+
sudo dnf install gcc-c++
46+
47+
# Ubuntu/Debian
48+
sudo apt install g++
49+
50+
# macOS (included with Xcode Command Line Tools)
51+
xcode-select --install
52+
```
53+
54+
### Running Tests
55+
56+
```bash
57+
# Start PostgreSQL container
58+
docker-compose -f tests/integration/docker-compose.yml up -d
59+
60+
# Wait for PostgreSQL to be ready (about 5 seconds)
61+
sleep 5
62+
63+
# Run all integration tests
64+
go test -v ./tests/integration/...
65+
66+
# Run without PostgreSQL (Duckgres-only mode)
67+
# Tests will automatically fall back if PostgreSQL isn't running
68+
go test -v ./tests/integration/...
69+
70+
# Stop PostgreSQL when done
71+
docker-compose -f tests/integration/docker-compose.yml down -v
72+
```
73+
74+
### Running Specific Test Categories
75+
76+
```bash
77+
# Data Query Language tests
78+
go test -v ./tests/integration/... -run TestDQL
79+
80+
# Data Definition Language tests
81+
go test -v ./tests/integration/... -run TestDDL
82+
83+
# Data Manipulation Language tests
84+
go test -v ./tests/integration/... -run TestDML
85+
86+
# Data type tests
87+
go test -v ./tests/integration/... -run TestTypes
88+
89+
# Function tests
90+
go test -v ./tests/integration/... -run TestFunctions
91+
92+
# System catalog tests
93+
go test -v ./tests/integration/... -run TestCatalog
94+
95+
# Session command tests
96+
go test -v ./tests/integration/... -run TestSession
97+
98+
# Wire protocol tests
99+
go test -v ./tests/integration/... -run TestProtocol
100+
101+
# Client tool compatibility tests
102+
go test -v ./tests/integration/clients/...
103+
```
104+
105+
## Test Coverage
106+
107+
| Category | Test Cases | Description |
108+
|----------|-----------|-------------|
109+
| **DQL** | ~150 | SELECT, WHERE, ORDER BY, GROUP BY, JOINs, CTEs, Window functions, Set operations |
110+
| **DDL** | ~50 | CREATE/ALTER/DROP TABLE, VIEW, INDEX, SCHEMA, constraints |
111+
| **DML** | ~45 | INSERT, UPDATE, DELETE, RETURNING, ON CONFLICT (UPSERT) |
112+
| **Types** | ~80 | Numeric, string, date/time, boolean, JSON, arrays, UUID, NULL handling |
113+
| **Functions** | ~180 | String, numeric, date/time, aggregate, JSON, array, conditional |
114+
| **Catalog** | ~50 | pg_catalog tables, information_schema, psql meta-commands |
115+
| **Session** | ~40 | SET, SHOW, RESET, transaction commands |
116+
| **Protocol** | ~40 | Simple query, prepared statements, transactions, error handling |
117+
| **Clients** | ~50 | Metabase, Grafana, Superset, Tableau, DBeaver, Fivetran, Airbyte, dbt |
118+
| **Total** | **~700** | |
119+
120+
## Directory Structure
121+
122+
```
123+
tests/integration/
124+
├── README.md # This file
125+
├── docker-compose.yml # PostgreSQL 16 container definition
126+
├── harness.go # Test harness for side-by-side testing
127+
├── compare.go # Result comparison utilities
128+
├── setup_test.go # Test initialization and helpers
129+
├── fixtures/
130+
│ ├── schema.sql # Test table definitions (18 tables, 3 views)
131+
│ └── data.sql # Test data
132+
├── dql_test.go # Data Query Language tests
133+
├── ddl_test.go # Data Definition Language tests
134+
├── dml_test.go # Data Manipulation Language tests
135+
├── types_test.go # Data type tests
136+
├── functions_test.go # Function compatibility tests
137+
├── catalog_test.go # pg_catalog/information_schema tests
138+
├── session_test.go # SET/SHOW/transaction tests
139+
├── protocol_test.go # Wire protocol tests
140+
└── clients/
141+
└── clients_test.go # BI/ETL tool compatibility tests
142+
```
143+
144+
## Comparison Strategy
145+
146+
The test suite uses **semantic equivalence** rather than byte-identical comparison:
147+
148+
- **Column names**: Case-insensitive comparison
149+
- **Row data**: Same values after type normalization
150+
- **Ordering**: Respects ORDER BY; otherwise treats results as sets
151+
- **NULL handling**: NULL == NULL for comparison purposes
152+
- **Numeric precision**: Float tolerance of 1e-9
153+
- **Timestamps**: Tolerance of 1 microsecond
154+
- **JSON**: Parsed and compared structurally
155+
156+
## Skipped Tests
157+
158+
Some tests are skipped with documented reasons:
159+
160+
| Skip Reason | Description |
161+
|-------------|-------------|
162+
| `SkipUnsupportedByDuckDB` | Feature not available in DuckDB |
163+
| `SkipDifferentBehavior` | Intentionally different (documented) |
164+
| `SkipOLTPFeature` | OLTP feature out of scope |
165+
| `SkipNetworkType` | Network types (INET, CIDR) not supported |
166+
| `SkipGeometricType` | Geometric types not supported |
167+
| `SkipRangeType` | Range types not supported |
168+
| `SkipTextSearch` | Full-text search not supported |
169+
170+
## Client Tool Compatibility
171+
172+
The test suite includes real queries used by popular tools:
173+
174+
### BI Tools
175+
- **Metabase**: Schema discovery, table introspection, column metadata
176+
- **Grafana**: Time column detection, table autocomplete
177+
- **Superset**: Table listing, column type discovery
178+
- **Tableau**: Schema/table/column discovery with full metadata
179+
- **DBeaver**: Catalog info, table and column metadata with descriptions
180+
181+
### ETL Tools
182+
- **Fivetran**: Schema sync, primary key detection, commented queries
183+
- **Airbyte**: Table discovery, column introspection
184+
- **dbt**: Relation existence checks, schema management, transactions
185+
186+
## Adding New Tests
187+
188+
### QueryTest Structure
189+
190+
```go
191+
tests := []QueryTest{
192+
{
193+
Name: "test_name", // Unique test name
194+
Query: "SELECT 1", // SQL to execute
195+
Skip: "", // Skip reason (empty = don't skip)
196+
DuckgresOnly: false, // true = skip PostgreSQL comparison
197+
ExpectError: false, // true = expect query to fail
198+
Options: DefaultCompareOptions(), // Comparison options
199+
},
200+
}
201+
runQueryTests(t, tests)
202+
```
203+
204+
### Custom Comparison Options
205+
206+
```go
207+
opts := CompareOptions{
208+
IgnoreColumnOrder: true, // Compare columns by name, not position
209+
IgnoreRowOrder: true, // Treat results as sets
210+
FloatTolerance: 1e-6, // Custom float tolerance
211+
IgnoreCase: true, // Case-insensitive string comparison
212+
}
213+
```
214+
215+
## Troubleshooting
216+
217+
### PostgreSQL container won't start
218+
219+
```bash
220+
# Check if port 35432 is in use
221+
lsof -i :35432
222+
223+
# View container logs
224+
docker-compose -f tests/integration/docker-compose.yml logs
225+
```
226+
227+
### Tests timeout
228+
229+
```bash
230+
# Increase test timeout
231+
go test -v -timeout 10m ./tests/integration/...
232+
```
233+
234+
### Connection refused errors
235+
236+
The test harness automatically retries connections. If you still see errors:
237+
238+
```bash
239+
# Ensure PostgreSQL is ready
240+
docker-compose -f tests/integration/docker-compose.yml exec postgres pg_isready
241+
242+
# Check Duckgres can start
243+
go test -v ./tests/integration/... -run TestProtocolSimpleQuery
244+
```
245+
246+
## Contributing
247+
248+
1. Add new test cases to the appropriate `*_test.go` file
249+
2. Use `DuckgresOnly: true` for queries that can't be compared to PostgreSQL
250+
3. Document skip reasons clearly
251+
4. Run the full suite before submitting:
252+
```bash
253+
go test -v ./tests/integration/...
254+
```

0 commit comments

Comments
 (0)