How do I change the buffer before copy command executed to airbyte_internal for redshift destination? #34787
Unanswered
meyerovb
asked this question in
Connector Questions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I'm doing a historical backfill into a redshift destination using s3 and copy commands. It is running copy into airbyte_internal schema tables too often. This is bad because this is redshift serverless... it's eating up credits and would be faster and more efficient if it ran copies with more and larger files. How do I make it store more and larger files in s3 before running the airbyte_internal copies to clear the 'buffer'?
Beta Was this translation helpful? Give feedback.
All reactions