-
Notifications
You must be signed in to change notification settings - Fork 8
Open
Labels
enhancementNew feature or requestNew feature or request
Description
We should add the ability to do simple DQ checks on s3 buckets. Say something like ...
results = tm.check_s3_uri(bucket='somebucket', prefix='some_prefix', check_type='files_exisit')
True
Could probably implement with boto3
Or something along those lines ...
- check files exist at some location
- count the number of files at some location
- check for empty files
- check for missing files over a date range
- check file names meet some regex
- etc.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request