Skip to content

Fix twin strategy implementation#278

Merged
simonpannek merged 29 commits intodremio:mainfrom
simonpannek:fix-twin-strategy
Mar 20, 2025
Merged

Fix twin strategy implementation#278
simonpannek merged 29 commits intodremio:mainfrom
simonpannek:fix-twin-strategy

Conversation

@simonpannek
Copy link
Copy Markdown
Member

@simonpannek simonpannek commented Feb 24, 2025

Summary

dbt assumes tables and views share locations, unlike Dremio that keeps tables and views separately. We address this conflict using the twin-strategy. Currently, the adapter's clone strategy seems to create a view for every table, even if there is no conflicting view to be replaced.

Description

With this change, we introduce a check that will only overwrite existing views, not create new ones on table creation when the clone strategy is enabled. When a view is created with a conflicting table, the view will always select the existing table.

Test Results

Due to this change, tests that queried the corresponding view after table creation (instead of the table directly) were failing due to this change. The schema of all affected tests have been adjusted.

Changelog

  • Only overwrite existing views in case of schema and name conflict
  • Adjust tests to query created tables directly instead of their matching views
  • Added a summary of what this PR accomplishes to CHANGELOG.md

@simonpannek simonpannek force-pushed the fix-twin-strategy branch 2 times, most recently from 2c2dc26 to 34962ef Compare March 7, 2025 15:49
@simonpannek simonpannek force-pushed the fix-twin-strategy branch 2 times, most recently from 5861848 to 0c82601 Compare March 10, 2025 19:15
@simonpannek simonpannek marked this pull request as ready for review March 11, 2025 10:05
@simonpannek simonpannek requested review from 99Lys and bcmeireles March 11, 2025 10:05
@simonpannek simonpannek marked this pull request as draft March 11, 2025 10:06
@simonpannek simonpannek changed the title Fix twin strategy Fix twin strategy implementation Mar 11, 2025
howareyouman
howareyouman previously approved these changes Mar 14, 2025
@simonpannek simonpannek force-pushed the fix-twin-strategy branch 2 times, most recently from d78ecb6 to a61d5f9 Compare March 20, 2025 09:42
howareyouman
howareyouman previously approved these changes Mar 20, 2025
Copy link
Copy Markdown

@howareyouman howareyouman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any other major changes from the last review.

@simonpannek
Copy link
Copy Markdown
Member Author

@howareyouman after rebasing, we had new failing tests, so these had to be adjusted. I also just realized we released 1.8.2 in the meanwhile, so this change needs to be moved to 1.8.3

@simonpannek simonpannek merged commit 8f8d4cd into dremio:main Mar 20, 2025
16 checks passed
@simonpannek simonpannek deleted the fix-twin-strategy branch March 20, 2025 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants