Twin Strategy Issue migration ⭐ #287
Description
With the incoming fix of the twin strategy, we'll be introducing a breaking change in version 1.8.3. (This had to be done in a minor version since our major versions are bound to the release cycle of dbt)
PR: #278
Context
Since Dremio handles views and tables in separate environments, we need to handle conflicts differently than other connectors. We address this conflict using the twin-strategy.
What changed
Due to an invalid implementation of the twin strategy, Dremio would always create a view when creating a table (matching name and schema).
This means, if a query creates a table, but then subsequently queries the matching view, the query would still go through. With this new update, a view will only be created (or overwritten in this case) if it already existed before.
Migration
If a query currently depends on a view that was only implicitly created by the adapter in the past, it is recommended to update this require to explicitly query the table in the data lake.
Example: Seed referencing using a source
A common issue we hit is referencing a seed using a source. By default, dbt will refer to the database (equivalent to Spaces in Dremio). In previous versions, this would work fine since the adapter would implicitly create a matching view for the table. Since this is not the case anymore with the new change, we'll have to explicitly target the table (see change below).
version: 2
sources:
- name: raw
+ database: "{{ target.datalake }}"
+ schema: "{{ target.root_path }}"
- schema: "{{ target.schema }}"
tables:
- name: seed
identifier: "{{ var('seed_name', 'base') }}"