Implement the updated Postgres 17 Merge+Returning feature for Postgres Destinations to allow for an Incremental + Merged Strategy #62195
walker-philips
started this conversation in
Connector Ideas and Features
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
The changes to Merge Returning in Postgres 17 could allow for an optimized merge strategy that handles inserts, updates, and deletes on records. As of now, data transferred into Postgres via incremental ensures an SCD-esque dataset which is suboptimal in certain circumstances as total parity between the source and destination is lost. By introducing a changing dimension, the Postgres Destination is almost certainly required to perform additional transformations to eliminate duplications as well as delete/inactivate rows no longer present in the source.
Beta Was this translation helpful? Give feedback.
All reactions