Stop build on failed test

[Junobijlard]

Hi all,

I'd like to discuss the execution order of model builds and model tests. TLDR; i propose to run tests immediately after a model build, and (optionally) stop the build of downstream models in case a test fails.

Situation

• Obviously, dbt has great support for tests
• In its current implementation, the dbt DAG first builds all models, and subsequently, runs all tests

Complication

• This means that you'll only catch errors once they are already there --> suppose something breaks in an upstream model. We first build all downstream models, only to figure out after all tables have been deployed that the data is off
• In some cases, I would prefer to not have fresh data, than having false data (e.g. financial reporting)
• In other cases, I would like to still proceed with the build (e.g. for an ML model)

Proposed solution

• Therefore, I'd like to see the DAG order changed; nodes in the DAG should consist of 1) model execution and 2) tests
• Using tags, it should be possible to mark a test as 'optional', which means that the downstream build can still continue even though the test has failed.
• This will also open the door for blue-green deployments or staging deployments (software engineering kind of staging environment, not the dbt staging model)

Example

version: 2

models:
  - name: orders
    columns:
      - name: order_id
        tests:
          - unique
          - not_null
      - name: status
        tests:
          - accepted_values:
              values: ['placed', 'shipped', 'completed', 'returned']
  - name: order_types
  - name: orders_by_month

Should compile to a DAG that looks like this:

[jaypeedevlin]

I think you're describing exactly the dbt build command!

[Junobijlard]

oh wow that's stupid haha. Thx for pointing out, somehow we are only using dbt run + dbt test