[source-s3] Enhancement: Allow for latest file flag #40543
michael-greene-gravie
started this conversation in
Connector Ideas and Features
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Requesting support for latest file flag in [source-s3]
Description
I would like to request a new feature for the [source-s3] connector in Airbyte. Currently, the connector allows us to ingest data from an S3 bucket by specifying a prefix or a list of files. However, it would be very useful if we could also specify a flag to ingest only the latest file in the bucket.
Feature Request
Add a new flag, let's call it
latestFile
, to the [source-s3] connector configuration. When this flag is set totrue
, the connector should only ingest the latest matching file.Example Usage
A full refresh file is being processed.
I am willing to work on this feature request.
Beta Was this translation helpful? Give feedback.
All reactions