How does Airbyte handle changes in replication start date? #32924
Unanswered
anteakar
asked this question in
Connector Questions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I am trying to backfill Universal Analytics data and send it to BigQuery. When I set a replication start date that is far in the past I get too many 503 errors (which I have noticed happen at random) which end up getting me blocked from the Universal Analytics API for 24h. To tackle this problem I want to get my data in increments, for example from 2018 to 2019, 2019 to 2020 etc., but the UA connector in Airbyte does not allow for a replication end date to be entered so any replication happens from the specified start date until the day the sync was started. To tackle this I tried to set the replication start date in the source to 2022, make the mode incremental append + dedupe and then after the sync is done go back to the source and change the start date to 2021. I was hoping that in this way I could backfill year by year. However, the connection does not seem to detect the date change in the source even when I update the source schema. Does anyone know how I could tackle this?
Beta Was this translation helpful? Give feedback.
All reactions