Migrate dbt producer from DuckDB to PostgreSQL

- Replace DuckDB adapter with PostgreSQL adapter in profiles.yml
- Update requirements.txt: dbt-postgres, psycopg2-binary
- Rename scenario csv_to_duckdb_local to csv_to_postgres_local
- Update scenario config.json lineage_level to postgres
- Add docker-compose.yml for local PostgreSQL container
- Update GitHub Actions workflow with PostgreSQL service container
- Fix README.md installation instructions (dbt-duckdb -> dbt-postgres)
- Update SPECIFICATION_COVERAGE_ANALYSIS.md to reference PostgreSQL

Tested locally: 22 OpenLineage events generated successfully with postgres namespace.

Signed-off-by: roller100 (BearingNode) <contact@bearingnode.com>

Author: roller100 (BearingNode)

.github/workflows/producer_dbt.yml

@@ -30,6 +30,22 @@ on:
 jobs:
   run-dbt-tests:
     runs-on: ubuntu-latest
+    
+    services:
+      postgres:
+        image: postgres:15-alpine
+        env:
+          POSTGRES_USER: testuser
+          POSTGRES_PASSWORD: testpass
+          POSTGRES_DB: dbt_test
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U testuser -d dbt_test"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+    
     steps:
       - name: Checkout code
         uses: actions/checkout@v4
@@ -56,7 +72,7 @@ jobs:
         run: |
           python -m pip install --upgrade pip
           pip install dbt-core==${{ inputs.dbt_release }}
-          pip install dbt-duckdb
+          pip install dbt-postgres
           pip install openlineage-dbt==${{ inputs.ol_release }}
           pip install -r producer/dbt/test_runner/requirements.txt
 

producer/dbt/README.md

@@ -85,9 +85,9 @@ To execute the test suite, you will need a local clone of the main [OpenLineage
     pip install -r test_runner/requirements.txt
     ```
 
-2.  **Install dbt and the DuckDB adapter**:
+2.  **Install dbt and the PostgreSQL adapter**:
     ```bash
-    pip install dbt-core dbt-duckdb
+    pip install dbt-core dbt-postgres
     ```
 
 3.  **Install the OpenLineage dbt integration**:

producer/dbt/SPECIFICATION_COVERAGE_ANALYSIS.md

@@ -6,7 +6,7 @@ This document analyzes the OpenLineage specification coverage achieved by our db
 ## Test Configuration
 - **OpenLineage Specification**: 2-0-2 (target specification)
 - **dbt-openlineage Implementation**: 1.37.0  
-- **Test Scenario**: CSV → dbt models → DuckDB (includes data quality tests)
+- **Test Scenario**: CSV → dbt models → PostgreSQL (includes data quality tests)
 - **Events Generated**: 20 events total
   - 3 dbt models (START/COMPLETE pairs)
   - 5 data quality test suites (START/COMPLETE pairs) 
@@ -31,7 +31,7 @@ This document analyzes the OpenLineage specification coverage achieved by our db
 
 | Facet | Status | Coverage | Notes |
 |-------|--------|----------|-------|
-| ✅ `processing_engine` | **TESTED** | Full validation | DuckDB processing engine captured |
+| ✅ `processing_engine` | **TESTED** | Full validation | PostgreSQL processing engine captured |
 | ✅ `parent` | **TESTED** | Full validation | Parent-child run relationships |
 | ✅ `dbt_run` | **TESTED** | Basic validation | dbt-specific run metadata (non-standard) |
 | ✅ `dbt_version` | **TESTED** | Basic validation | dbt version information (non-standard) |
@@ -92,7 +92,7 @@ This document analyzes the OpenLineage specification coverage achieved by our db
 
 ### 🏗️ Infrastructure Constraints  
 - **Local File Transport**: Missing network-based transport scenarios
-- **DuckDB Only**: Missing other database-specific facets
+- **PostgreSQL Only**: Missing other database-specific facets
 - **No CI/CD Context**: Missing environment variables, build metadata
 - **No Version Control**: Missing source code location tracking
 

producer/dbt/docker-compose.yml

@@ -0,0 +1,23 @@
+version: '3.8'
+
+services:
+  postgres:
+    image: postgres:15-alpine
+    container_name: dbt-test-postgres
+    environment:
+      POSTGRES_USER: testuser
+      POSTGRES_PASSWORD: testpass
+      POSTGRES_DB: dbt_test
+    ports:
+      - "5432:5432"
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U testuser -d dbt_test"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+    volumes:
+      - postgres_data:/var/lib/postgresql/data
+
+volumes:
+  postgres_data:
+    name: dbt_test_postgres_data

producer/dbt/runner/profiles.yml

@@ -2,7 +2,11 @@ openlineage_compatibility_test:
   target: dev
   outputs:
     dev:
-      type: duckdb
-      path: './openlineage_test.duckdb'
-      schema: main
-      threads: 1
+      type: postgres
+      host: "{{ env_var('DBT_POSTGRES_HOST', 'localhost') }}"
+      port: "{{ env_var('DBT_POSTGRES_PORT', '5432') | as_number }}"
+      user: "{{ env_var('DBT_POSTGRES_USER', 'testuser') }}"
+      password: "{{ env_var('DBT_POSTGRES_PASSWORD', 'testpass') }}"
+      dbname: "{{ env_var('DBT_POSTGRES_DB', 'dbt_test') }}"
+      schema: "{{ env_var('DBT_POSTGRES_SCHEMA', 'main') }}"
+      threads: 4

…/scenarios/csv_to_duckdb_local/README.md …cenarios/csv_to_postgres_local/README.mdproducer/dbt/scenarios/csv_to_duckdb_local/README.md renamed to producer/dbt/scenarios/csv_to_postgres_local/README.md producer/dbt/scenarios/csv_to_duckdb_local/README.md renamed to producer/dbt/scenarios/csv_to_postgres_local/README.md

@@ -16,7 +16,7 @@
           "max_version": "2-0-2",
           "min_version": "1.0.0",
           "lineage_level": {
-            "duckdb": ["dataset", "column"]
+            "postgres": ["dataset", "column"]
           }
         }
       },
@@ -28,7 +28,7 @@
           "max_version": "2-0-2",
           "min_version": "1.0.0",
           "lineage_level": {
-            "duckdb": ["dataset"]
+            "postgres": ["dataset"]
           }
         }
       },
@@ -40,7 +40,7 @@
           "max_version": "2-0-2", 
           "min_version": "1.0.0",
           "lineage_level": {
-            "duckdb": ["dataset", "transformation"]
+            "postgres": ["dataset", "transformation"]
           }
         }
       },
@@ -52,7 +52,7 @@
           "max_version": "2-0-2",
           "min_version": "1.0.0", 
           "lineage_level": {
-            "duckdb": ["column", "transformation"]
+            "postgres": ["column", "transformation"]
           }
         }
       }