Skip to content

Commit 222dd69

Browse files
Outline new process
1 parent d83fda1 commit 222dd69

File tree

1 file changed

+43
-5
lines changed

1 file changed

+43
-5
lines changed

docs/integrations/destinations/snowflake-migrations.md

Lines changed: 43 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,18 +4,56 @@ import MigrationGuide from '@site/static/_migration_guides_upgrade_guide.md';
44

55
## Upgrading to 4.0.0
66

7-
This version upgrades Destination Snowflake to the [Direct-Load](/platform/using-airbyte/core-concepts/direct-load-tables) paradigm, which improves performance and reduces warehouse spend. If you have unusual requirements around record visibility or schema evolution, read that document for more information about how direct-load differs from Typing and Deduping.
7+
This version upgrades the Snowflake destination from using [typing and deduping](/platform/using-airbyte/core-concepts/typing-deduping) to [direct loading](/platform/using-airbyte/core-concepts/direct-load-tables). This upgrade improves performance and reduces warehouse spend. If you have unusual requirements around record visibility or schema evolution, read the documentation for those methodologies for more information about how direct loading differs from typing and deduping.
88

9-
This version also adds an option to enable CDC deletions as soft-deletes.
9+
This version also adds an option to enable CDC deletions as soft deletes.
1010

11-
If you do not interact with the raw tables, you can safely upgrade. There is no breakage for this usecase.
11+
### Decide how to handle raw tables
1212

13-
If you _only_ interact with the raw tables, make sure that you have the `Disable Final Tables` option enabled before upgrading. This will automatically enable the `Legacy raw tables` option after upgrading.
13+
The exact steps to begin the migration depend on whether you interact with Airbyte's raw tables in Snowflake.
1414

15-
If you interact with both the raw _and_ final tables, this usecase will no longer be directly supported. You must create two connectors (one with `Disable Final Tables` enabled, and one with it disabled) and run two connections in parallel.
15+
#### If you don't interact with raw tables
16+
17+
If you don't interact with the raw tables, you can safely upgrade. There's no breaking change for you.
18+
19+
#### If you only interact with raw tables
20+
21+
If you _only_ interact with the raw tables, enable the `Disable Final Tables` option before upgrading. This enables the `Legacy raw tables` option after upgrading.
22+
23+
1. In the navigation bar, click **Destinations**.
24+
25+
2. Open your **Snowflake** connector.
26+
27+
3. <!-- Devin: fill out the rest of these steps to use Disable Final Tables from Snowflake v3 -->
28+
29+
#### If you interact with both raw and final tables
30+
31+
If you interact with both the raw _and_ final tables, this use case is no longer supported. Instead, create two connectors. One with `Disable Final Tables` turned on, and one with it turned off. Starting now, you must run the two connections in parallel.
32+
33+
1. In the navigation bar, click **Destinations**.
34+
35+
2. For each Snowflake destination you have, add an identical second Snowflake destination.
36+
37+
3. Ensure each pair of Snowflake connectors have opposite settings for <!-- Devin: fill out the rest of this step to use Disable Final Tables from Snowflake v3 -->
38+
39+
4. <!-- Devin: fill out the rest of this process -->
1640

1741
<MigrationGuide />
1842

43+
### Optional: clean up legacy raw tables
44+
45+
The version 4.0 connector doesn't automatically remove tables created by earlier versions. After upgrading to version 4 and verifying your data, you can optionally remove the old raw tables.
46+
47+
For most users, You can find the raw tables <!-- Devin: where? -->. The names match the pattern <!-- Devin: what are the tables and schemas called? -->
48+
49+
:::note
50+
Version 4 of the Snowfalke destination uses the `airbyte_internal` database for temporary scratch space (for example, streams running in dedup mode, truncate refreshes, and overwrite syncs). Dropping the entire `airbyte_internal database` can interrupt active syncs and cause data loss. Only drop the specific raw tables you no longer need.
51+
:::
52+
53+
To remove the old raw tables:
54+
55+
<!-- Devin: SQL query to delete a raw table in Snowflake -->
56+
1957
## Upgrading to 3.0.0
2058

2159
This version introduces [Destinations V2](/release_notes/upgrading_to_destinations_v2/#what-is-destinations-v2), which provides better error handling, incremental delivery of data for large syncs, and improved final table structures. To review the breaking changes, and how to upgrade, see [here](/release_notes/upgrading_to_destinations_v2/#quick-start-to-upgrading). These changes will likely require updates to downstream dbt / SQL models, which we walk through [here](/release_notes/upgrading_to_destinations_v2/#updating-downstream-transformations). Selecting `Upgrade` will upgrade **all** connections using this destination at their next sync. You can manually sync existing connections prior to the next scheduled sync to start the upgrade early.

0 commit comments

Comments
 (0)