Skip to content

[BUG] Selecting from columns in a snapshot but no longer in model upstream of snapshot raises compilation errors #1141

@b-per

Description

@b-per

Describe the bug

Raised on Slack

Today if a snapshot contains a column that is no longer in the snapshotted model then models downstream of the snapshot can't select this column.

With the example of model1 > snap_model1 > model2

If a column called to_be_deleted was in model1
and this model was snapshotted (so snap_model1 contains the column)
and model2 reads the column to_be_deleted from snap_model1

Fusion will stop compiling as soon as to_be_deleted is removed from model1, even if it is still in snap_model1

Fusion should not infer the schema of snap_model1 from model1.

  • if the snapshot exists, then we should use a schema that would be a combination of the columns of model1 and the columns of snap_model1. That way, compilation would work if a column is added or removed in model1 and we try to read those columns from model2
  • if the snapshot doesn't exist, then we can infer the schema from model1, adding the columns automatically created by the snapshot

What version of dbt Fusion is this bug in? (find out by running dbt --version)
Preview 86

To Reproduce
This is self explanatory as described in the use case above.
It requires having 1 model > 1 snapshot > 1 model and removing a column from the first model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ConformancebugSomething isn't workinghas-reproHas a reproducible example

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions