From 705d340fe99513031dc0a1d1edc4387f8c4afbb2 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Wed, 19 Nov 2025 00:53:53 +0000 Subject: [PATCH 1/3] docs: expand Snowflake v4 migration guide with detailed instructions - Add step-by-step UI instructions for enabling Disable Final Tables in v3 - Provide detailed guidance for users with both raw and final tables - Document raw table naming patterns for v2/v3 and v4 - Add SQL queries for finding and dropping legacy raw tables - Fix typo: Snowfalke -> Snowflake - Fix terminology: database -> schema for airbyte_internal Co-Authored-By: ian.alton@airbyte.io --- .../destinations/snowflake-migrations.md | 59 +++++++++++++++++-- 1 file changed, 53 insertions(+), 6 deletions(-) diff --git a/docs/integrations/destinations/snowflake-migrations.md b/docs/integrations/destinations/snowflake-migrations.md index fecfe494bd09..4a5293befa30 100644 --- a/docs/integrations/destinations/snowflake-migrations.md +++ b/docs/integrations/destinations/snowflake-migrations.md @@ -24,7 +24,17 @@ If you _only_ interact with the raw tables, enable the `Disable Final Tables` op 2. Open your **Snowflake** connector. -3. +3. Click **Edit configuration**. + +4. Open the **Advanced** section. + +5. Turn on **Disable Final Tables**. + +6. Click **Save**. + +:::note +After upgrading to version 4, this setting appears as **Legacy raw tables** and should remain enabled. +::: #### If you interact with both raw and final tables @@ -34,9 +44,19 @@ If you interact with both the raw _and_ final tables, this use case is no longer 2. For each Snowflake destination you have, add an identical second Snowflake destination. -3. Ensure each pair of Snowflake connectors have opposite settings for +3. Ensure each pair of Snowflake connectors have opposite settings for **Disable Final Tables** (in version 3) or **Legacy raw tables** (in version 4). One connector should have this setting enabled, and the other should have it disabled. + +4. Configure distinct default schemas for each destination to avoid table name collisions. + +5. Update your connections to point to the appropriate destination: + - Connections that need raw tables only should target the destination with **Disable Final Tables** or **Legacy raw tables** enabled. + - Connections that need final tables should target the destination with this setting disabled. -4. +6. Run test syncs on both destinations to verify outputs: + - The raw-only destination should write only to the internal schema (default `airbyte_internal`). + - The standard destination should write only final tables to the target schema. + +7. After verifying that both destinations work correctly, continue running both connections in parallel going forward. @@ -44,15 +64,42 @@ If you interact with both the raw _and_ final tables, this use case is no longer The version 4.0 connector doesn't automatically remove tables created by earlier versions. After upgrading to version 4 and verifying your data, you can optionally remove the old raw tables. -For most users, You can find the raw tables . The names match the pattern +For most users, you can find the raw tables in the schema configured as **Airbyte Internal Table Dataset Name** (which defaults to `airbyte_internal`). If you customized this setting, look in that schema instead. + +The table names match these patterns depending on which version created them: + +- **Version 2/3 (Typing and Deduping)**: `raw_{namespace}__{stream}` (for example, `airbyte_internal.raw_public__users`) +- **Version 4 (Legacy raw tables mode)**: `{namespace}_raw__stream_{stream}` (for example, `airbyte_internal.public_raw__stream__users`) + +Note: The number of underscores between `raw` and `stream` may vary depending on the longest underscore sequence in your namespace and stream names. :::note -Version 4 of the Snowfalke destination uses the `airbyte_internal` database for temporary scratch space (for example, streams running in dedup mode, truncate refreshes, and overwrite syncs). Dropping the entire `airbyte_internal database` can interrupt active syncs and cause data loss. Only drop the specific raw tables you no longer need. +Version 4 of the Snowflake destination uses the `airbyte_internal` schema for temporary scratch space (for example, streams running in dedup mode, truncate refreshes, and overwrite syncs). Dropping the entire `airbyte_internal` schema can interrupt active syncs and cause data loss. Only drop the specific raw tables you no longer need. ::: To remove the old raw tables: - +1. **Pause or allow active syncs to complete** before dropping any tables to avoid interrupting data transfers. + +2. **List candidate raw tables** to identify which tables to remove: + + ```sql + -- For Version 2/3 raw tables: + SHOW TABLES IN SCHEMA . LIKE 'RAW\_%'; + + -- For Version 4 legacy raw tables: + SHOW TABLES IN SCHEMA . LIKE '%_RAW__STREAM_%'; + ``` + + Replace `` with your Snowflake database name and `` with your internal schema name (default `airbyte_internal`). + +3. **Drop specific raw tables** you no longer need: + + ```sql + DROP TABLE IF EXISTS ..; + ``` + + Replace `` with the specific table name you want to remove. Use fully qualified names (database.schema.table) to avoid ambiguity. ## Upgrading to 3.0.0 From 2e8bdd510430129e0264fe86f97a38693f8d789f Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Wed, 19 Nov 2025 23:03:05 +0000 Subject: [PATCH 2/3] docs: address PR feedback on Snowflake migration guide - Remove 'should' wording, assume success (comment 2543815075) - Use V3 terminology only, remove V4 references before upgrade (comments 2543817615, 2543818089) - Elaborate on schema configuration with concrete examples (comment 2543819122) - Fix markdownlint trailing spaces in SQL code blocks - Change 'enabled/disabled' to 'turned on/off' per Vale style guide Co-Authored-By: ian.alton@airbyte.io --- .../destinations/snowflake-migrations.md | 22 ++++++++++++------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/docs/integrations/destinations/snowflake-migrations.md b/docs/integrations/destinations/snowflake-migrations.md index 4a5293befa30..fe89746c2e7b 100644 --- a/docs/integrations/destinations/snowflake-migrations.md +++ b/docs/integrations/destinations/snowflake-migrations.md @@ -33,7 +33,7 @@ If you _only_ interact with the raw tables, enable the `Disable Final Tables` op 6. Click **Save**. :::note -After upgrading to version 4, this setting appears as **Legacy raw tables** and should remain enabled. +After upgrading to version 4, this setting appears as **Legacy raw tables** and remains enabled. ::: #### If you interact with both raw and final tables @@ -44,13 +44,19 @@ If you interact with both the raw _and_ final tables, this use case is no longer 2. For each Snowflake destination you have, add an identical second Snowflake destination. -3. Ensure each pair of Snowflake connectors have opposite settings for **Disable Final Tables** (in version 3) or **Legacy raw tables** (in version 4). One connector should have this setting enabled, and the other should have it disabled. +3. Ensure each pair of Snowflake connectors have opposite settings for **Disable Final Tables**. One connector should have this setting turned on, and the other should have it turned off. -4. Configure distinct default schemas for each destination to avoid table name collisions. +4. Configure distinct default schemas for each destination to avoid table name collisions: + - For the destination that will create final tables, set a distinct **Schema** in the Snowflake destination configuration (for example, `ANALYTICS_V4`). This is where final tables will be written. + - For the raw-only destination (with **Disable Final Tables** turned on), set a distinct **Airbyte Internal Table Dataset Name** under the **Advanced** section (for example, `AIRBYTE_INTERNAL_RAW`). This is where raw tables will be written. + - Example configuration: + - Destination A (final tables): Schema = `ANALYTICS_V4`, Airbyte Internal Table Dataset Name = `AIRBYTE_INTERNAL` + - Destination B (raw-only): Airbyte Internal Table Dataset Name = `AIRBYTE_INTERNAL_RAW` + - Using distinct schemas prevents table name collisions when running both destinations in parallel. 5. Update your connections to point to the appropriate destination: - - Connections that need raw tables only should target the destination with **Disable Final Tables** or **Legacy raw tables** enabled. - - Connections that need final tables should target the destination with this setting disabled. + - Connections that need raw tables only should target the destination with **Disable Final Tables** turned on. + - Connections that need final tables should target the destination with this setting turned off. 6. Run test syncs on both destinations to verify outputs: - The raw-only destination should write only to the internal schema (default `airbyte_internal`). @@ -86,11 +92,11 @@ To remove the old raw tables: ```sql -- For Version 2/3 raw tables: SHOW TABLES IN SCHEMA . LIKE 'RAW\_%'; - + -- For Version 4 legacy raw tables: SHOW TABLES IN SCHEMA . LIKE '%_RAW__STREAM_%'; ``` - + Replace `` with your Snowflake database name and `` with your internal schema name (default `airbyte_internal`). 3. **Drop specific raw tables** you no longer need: @@ -98,7 +104,7 @@ To remove the old raw tables: ```sql DROP TABLE IF EXISTS ..; ``` - + Replace `` with the specific table name you want to remove. Use fully qualified names (database.schema.table) to avoid ambiguity. ## Upgrading to 3.0.0 From 0735277382bf85af3f62dfd1e6af2ff8b4a5b55c Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Wed, 19 Nov 2025 23:09:20 +0000 Subject: [PATCH 3/3] docs: fix UI steps for enabling Disable Final Tables in v3 - Remove 'Edit configuration' step (defaults to Settings tab) - Keep 'Advanced' section (verified in v3 spec) - Change 'Save' to 'Test and save' (correct button label) Addresses comment 2543835428 Co-Authored-By: ian.alton@airbyte.io --- docs/integrations/destinations/snowflake-migrations.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/docs/integrations/destinations/snowflake-migrations.md b/docs/integrations/destinations/snowflake-migrations.md index fe89746c2e7b..2daf1aa397b6 100644 --- a/docs/integrations/destinations/snowflake-migrations.md +++ b/docs/integrations/destinations/snowflake-migrations.md @@ -24,13 +24,11 @@ If you _only_ interact with the raw tables, enable the `Disable Final Tables` op 2. Open your **Snowflake** connector. -3. Click **Edit configuration**. +3. Open the **Advanced** section. -4. Open the **Advanced** section. +4. Turn on **Disable Final Tables**. -5. Turn on **Disable Final Tables**. - -6. Click **Save**. +5. Click **Test and save**. :::note After upgrading to version 4, this setting appears as **Legacy raw tables** and remains enabled.