Add a Lookback Window on Cursor Based Postgres Connection #52667
tmas-definitive
started this conversation in
Connector Ideas and Features
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
In the current design of the postgres source, a lookback window cannot be specified which would be extremely helpful in a cursor based replication strategy if a specific window of data needed to be replayed. I realize WAL or Xmin can probably handle this already, but if there are constraints to using cursor based (ie. pulling data from a read replica)
So now the query roughly generated on a cursor based run would be:
SELECT <columns> FROM <stream_source> WHERE cursor_field > {cursor_value}
Would like it to look something like
SELECT <columns> FROM <stream_source> WHERE cursor_field > {cursor_value} - {lookback}
Beta Was this translation helpful? Give feedback.
All reactions