How does "Use MERGE for De-duplication of final tables" work in Snowflake destination? #47219
hongbo-miao
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I am trying to understand how "Use MERGE for De-duplication of final tables" works in Snowflake desitination.
If source data before is
and changed to
And this stream is using "Incremental| Append + Deduped".
Here is my understnading:
When "Use MERGE for De-duplication of final tables" is disabled, the Snowflake destination will change to
When "Use MERGE for De-duplication of final tables" is enabled, will the Snowflake destination will change to
Am I understanding correctly? Thanks!
UPDATE 1:
Hmm I may be wrong. I saw source code.
When "Use MERGE for De-duplication of final tables" disabled, it calls
insertRecordsDedupe
. InsideinsertRecordsDedupe
, it callscdcDeletes
. So I assume it also deletes data.Hmm so now I am not fully sure about their difference.
Beta Was this translation helpful? Give feedback.
All reactions