docs(dbt): Document custom facets and integrate coverage analysis

Final documentation updates for PostgreSQL migration:

1. SPECIFICATION_COVERAGE_ANALYSIS.md:
   - Updated test configuration (PostgreSQL 15, 22 events, matrix testing)
   - Added comprehensive 'Known Validation Warnings' section
   - Documented dbt_version and dbt_run custom facets
   - Explained why warnings occur (vendor extensions vs official spec)
   - Clarified impact: tests pass, events valid, warnings expected
   - Listed resolution options and current workaround status

2. README.md:
   - Distinguished local vs GitHub Actions testing workflows
   - Added 'Custom dbt Facets and Validation Warnings' section
   - Cross-referenced SPECIFICATION_COVERAGE_ANALYSIS.md at two key points
   - Clarified that validation warnings are expected behavior

These docs ensure contributors understand:
- The difference between local Docker Compose and CI/CD testing
- Why dbt events generate validation warnings (custom facets)
- That warnings are documented, expected, and acceptable
- Where to find detailed technical analysis

Ready for upstream PR to OpenLineage compatibility-tests repo.

Signed-off-by: roller100 (BearingNode) <[email protected]>

Author: roller100 (BearingNode)

producer/dbt/README.md

@@ -52,7 +52,12 @@ This test validates that the `openlineage-dbt` integration correctly generates O
     -   `dataQualityAssertions` (for dbt tests)
 -   **Specification Compliance**: Events are validated against the OpenLineage specification schema (version `2-0-2`).
 
-A detailed, facet-by-facet analysis of specification coverage is available in `SPECIFICATION_COVERAGE_ANALYSIS.md`.
+**For detailed coverage analysis**, see **[`SPECIFICATION_COVERAGE_ANALYSIS.md`](./SPECIFICATION_COVERAGE_ANALYSIS.md)** which provides:
+- Comprehensive facet-by-facet coverage breakdown (39% overall specification coverage)
+- Detailed explanation of custom dbt facets and validation warnings
+- Analysis of what's tested vs. what's not tested and why
+- Recommendations for future coverage improvements
+- Resolution status for known validation warnings
 
 ## Test Structure
 
@@ -75,54 +80,174 @@ producer/dbt/
 
 ## How to Run the Tests
 
-To execute the test suite, you will need a local clone of the main [OpenLineage repository](https://github.com/OpenLineage/OpenLineage), as the validation tool requires access to the specification files.
+There are two primary ways to run the dbt compatibility tests: **locally for development and debugging**, or via **GitHub Actions for automated CI/CD validation**. Both approaches use the same underlying test framework but differ in their database setup and execution environment.
 
-### Prerequisites
+### Running Tests via GitHub Actions (Automated CI/CD)
 
-1.  **Install Python Dependencies**:
+**This is the standard, automated test runner for the repository and community.**
+
+GitHub Actions provides the canonical testing environment with:
+- PostgreSQL 15 service container (automatically provisioned)
+- Matrix testing across multiple dbt and OpenLineage versions
+- Automated event validation against OpenLineage specifications
+- Integration with the repository's reporting and compatibility tracking
+
+#### Triggering GitHub Actions Workflows
+
+1. **Automatic Trigger on Pull Requests**: The workflow runs automatically when changes are detected in `producer/dbt/` paths.
+
+2. **Manual Trigger via Workflow Dispatch**:
+   ```bash
+   # Trigger for specific branch
+   gh workflow run main_pr.yml --ref feature/your-branch -f components="dbt"
+   
+   # Watch the run
+   gh run watch
+   ```
+
+3. **Via Pull Request**: Opening a PR that modifies dbt producer files will automatically trigger the test suite.
+
+The GitHub Actions workflow:
+- Provisions a PostgreSQL 15 container with health checks
+- Installs `dbt-core`, `dbt-postgres`, and `openlineage-dbt` at specified versions
+- Executes all scenarios defined in `scenarios/`
+- Validates events against OpenLineage JSON schemas
+- Generates compatibility reports and uploads artifacts
+
+**Configuration**: See `.github/workflows/producer_dbt.yml` for the complete workflow definition.
+
+---
+
+### Running Tests Locally (Development & Debugging)
+
+**Use this approach for iterative development, debugging, and testing changes before pushing to GitHub.**
+
+Local testing provides:
+- Faster feedback loops for development
+- Direct access to event files and logs
+- Ability to inspect database state
+- Control over specific test scenarios
+
+#### Prerequisites
+
+1.  **Start PostgreSQL Container**:
     ```bash
     # From the producer/dbt/ directory
+    docker-compose up -d
+    
+    # Verify container is healthy
+    docker-compose ps
+    ```
+
+2.  **Install Python Dependencies**:
+    ```bash
+    # Activate virtual environment (recommended)
+    python -m venv venv
+    source venv/bin/activate  # On Windows: venv\Scripts\activate
+    
+    # Install requirements
     pip install -r test_runner/requirements.txt
     ```
 
-2.  **Install dbt and the PostgreSQL adapter**:
+3.  **Install dbt and the PostgreSQL adapter**:
     ```bash
     pip install dbt-core dbt-postgres
     ```
 
-3.  **Install the OpenLineage dbt integration**:
+4.  **Install the OpenLineage dbt integration**:
     ```bash
     pip install openlineage-dbt
     ```
 
-### Execution
+5.  **Verify dbt Connection**:
+    ```bash
+    cd runner/
+    dbt debug
+    cd ..
+    ```
+
+#### Local Execution Options
 
-Run the main test script, providing the path to your local OpenLineage repository.
+**Option 1: Using the Test Runner CLI (Recommended)**
 
-#### Basic Example
-This command runs the test suite with default settings, validating against the `2-0-2` OpenLineage release and saving events to the `events/` directory.
+The test runner CLI provides the same orchestration used in GitHub Actions:
 
 ```bash
-# Example assuming the OpenLineage repo is cloned in a sibling directory
-./run_dbt_tests.sh --openlineage-directory ../OpenLineage
+# Run a specific scenario
+python test_runner/cli.py run-scenario \
+  --scenario csv_to_postgres_local \
+  --output-dir ./test_output/$(date +%s)
+
+# List available scenarios
+python test_runner/cli.py list-scenarios
 ```
 
-#### Full Example
-This command demonstrates how to override the default settings by specifying all available arguments.
+**Option 2: Direct dbt-ol Execution (For debugging)**
+
+For fine-grained control and debugging, run `dbt-ol` commands directly:
+
+```bash
+cd runner/
+
+# Generate events for seed operation
+dbt-ol seed
+
+# Generate events for model execution
+dbt-ol run
+
+# Generate events for tests
+dbt-ol test
+
+# Inspect generated events
+cat ../events/openlineage_events.jsonl | jq '.'
+```
+
+**Option 3: Legacy Shell Script (Deprecated)**
+
+The `run_dbt_tests.sh` script is deprecated but still available:
 
 ```bash
 ./run_dbt_tests.sh \
-  --openlineage-directory /path/to/your/OpenLineage \
-  --producer-output-events-dir /tmp/dbt_events \
+  --openlineage-directory /path/to/OpenLineage \
+  --producer-output-events-dir ./events \
   --openlineage-release 2-0-2 \
-  --report-path /tmp/dbt_report.json
+  --report-path ./dbt_report.json
+```
+
+#### Local vs. GitHub Actions: Key Differences
+
+| Aspect | Local Testing | GitHub Actions |
+|--------|---------------|----------------|
+| **Database** | Docker Compose (manual start) | PostgreSQL service container (auto-provisioned) |
+| **Environment** | Uses local environment variables from `profiles.yml` | Uses workflow-defined environment variables |
+| **Event Output** | Writes to `events/openlineage_events.jsonl` by default | Writes to temporary directory defined by workflow |
+| **Validation** | Manual inspection or via test runner CLI | Automated validation against OpenLineage schemas |
+| **Use Case** | Development, debugging, local verification | CI/CD, PR validation, compatibility reporting |
+| **Cleanup** | Manual (`docker-compose down -v`) | Automatic container cleanup |
+
+#### Cleaning Up Local Environment
+
+```bash
+# Stop PostgreSQL container
+docker-compose down
+
+# Remove PostgreSQL data volume (clean slate)
+docker-compose down -v
+
+# Remove generated event files
+rm -rf events/*.jsonl test_output/
 ```
 
-### Command-Line Arguments
--   `--openlineage-directory` (**Required**): Path to the root of a local clone of the OpenLineage repository, which contains the `spec/` directory.
--   `--producer-output-events-dir`: Directory where generated OpenLineage events will be saved. (Default: `events/`)
--   `--openlineage-release`: The OpenLineage release version to validate against. (Default: `2-0-2`)
--   `--report-path`: Path where the final JSON test report will be generated. (Default: `../dbt_producer_report.json`)
+---
+
+### Command-Line Arguments (Legacy Script)
+
+For the deprecated `run_dbt_tests.sh` script:
+
+-   `--openlineage-directory` (**Required**): Path to a local clone of the OpenLineage repository
+-   `--producer-output-events-dir`: Directory for generated OpenLineage events (Default: `events/`)
+-   `--openlineage-release`: OpenLineage release version to validate against (Default: `2-0-2`)
+-   `--report-path`: Path for the final JSON test report (Default: `../dbt_producer_report.json`)
 
 ## Important dbt Integration Notes
 
@@ -135,6 +260,35 @@ This integration has several nuances that are important to understand when analy
 -   The availability of certain dbt-specific facets may depend on the version of `dbt-core` being used.
 -   The file transport configuration in `openlineage.yml` directly controls the location and format of the event output.
 
+### Custom dbt Facets and Validation Warnings
+
+**The dbt integration emits custom facets that generate expected validation warnings:**
+
+The `openlineage-dbt` integration adds vendor-specific facets to OpenLineage events that are **not part of the official OpenLineage specification**:
+
+1. **`dbt_version`** - Captures the dbt-core version
+2. **`dbt_run`** - Captures dbt execution metadata (invocation_id, profile_name, project_name, etc.)
+
+These facets:
+- ✅ Have valid schema definitions in the OpenLineage repository
+- ✅ Provide valuable dbt-specific context for lineage consumers
+- ⚠️ Generate validation warnings: `"facet type dbt_version not recognized"` and `"facet type dbt_run not recognized"`
+- ℹ️ Are **expected behavior** for vendor-specific OpenLineage extensions
+
+**Impact on Test Results:**
+- All dbt operations complete successfully (seed, run, test)
+- All events are generated with correct OpenLineage structure
+- Core facets (schema, dataSource, sql, columnLineage, etc.) validate successfully
+- Custom dbt facets trigger warnings during schema validation but do **not indicate test failure**
+
+These warnings are **documented and accepted** as expected behavior. 
+
+**📊 For complete technical details**, see **[`SPECIFICATION_COVERAGE_ANALYSIS.md`](./SPECIFICATION_COVERAGE_ANALYSIS.md)** which documents:
+- The exact structure and purpose of `dbt_version` and `dbt_run` facets
+- Why validation warnings occur (vendor extensions vs. official spec)
+- Impact assessment on test results
+- Current workarounds and long-term resolution options
+
 ## Future Enhancements
 
 To support community discussions around forward and backward compatibility, the `future/` directory contains design documents exploring a potential approach to multi-spec and multi-implementation version testing.

producer/dbt/SPECIFICATION_COVERAGE_ANALYSIS.md

@@ -5,12 +5,52 @@ This document analyzes the OpenLineage specification coverage achieved by our db
 
 ## Test Configuration
 - **OpenLineage Specification**: 2-0-2 (target specification)
-- **dbt-openlineage Implementation**: 1.37.0  
+- **dbt-openlineage Implementation**: 1.39.0 / 1.23.0 (matrix tested)
+- **Database**: PostgreSQL 15 (migrated from DuckDB)
 - **Test Scenario**: CSV → dbt models → PostgreSQL (includes data quality tests)
-- **Events Generated**: 20 events total
+- **Events Generated**: 22 events total
   - 3 dbt models (START/COMPLETE pairs)
   - 5 data quality test suites (START/COMPLETE pairs) 
   - 1 job orchestration wrapper (START/COMPLETE)
+  - Additional seed operations
+
+## ⚠️ Known Validation Warnings
+
+The dbt integration emits **custom facets that are not part of the official OpenLineage specification**. These generate validation warnings but are **expected and acceptable**:
+
+### Custom dbt Facets:
+1. **`dbt_version`** (Run Facet)
+   - **Purpose**: Captures the version of dbt-core being used
+   - **Schema**: `dbt-version-run-facet.json`
+   - **Example**: `{"version": "1.10.15"}`
+   - **Validation Warning**: `"$.run.facets.dbt_version facet type dbt_version not recognized"`
+
+2. **`dbt_run`** (Run Facet)
+   - **Purpose**: Captures dbt-specific execution metadata
+   - **Schema**: `dbt-run-run-facet.json`
+   - **Fields**: `dbt_runtime`, `invocation_id`, `profile_name`, `project_name`, `project_version`
+   - **Validation Warning**: `"$.run.facets.dbt_run facet type dbt_run not recognized"`
+
+### Why These Warnings Occur:
+- The OpenLineage specification validator checks against the **official spec schemas**
+- Custom vendor-specific facets (like dbt's) are **extensions** to the core spec
+- These facets have valid schema URLs but are not included in the official OpenLineage specification
+- The warnings indicate the validator found facets it doesn't recognize, **not that the events are invalid**
+
+### Impact on Testing:
+- ✅ **All dbt operations execute successfully** (seed, run, test)
+- ✅ **All 22 events are generated correctly** with proper structure
+- ✅ **Core OpenLineage facets validate successfully** (schema, dataSource, sql, etc.)
+- ⚠️ **Custom dbt facets generate warnings** during schema validation
+- ℹ️ **This is expected behavior** for vendor-specific extensions to OpenLineage
+
+### Resolution Status:
+- **Current State**: Warnings are documented and accepted as expected behavior
+- **Workaround**: `fail-for-new-failures` temporarily disabled in GitHub Actions for feature branch testing
+- **Long-term Options**:
+  1. Update validation to allow custom facets with valid schema URLs
+  2. Propose dbt facets for inclusion in official OpenLineage specification
+  3. Accept warnings as documented known behavior after merge to main
 
 ## Facet Coverage Analysis