Skip to content

Commit 3f581e6

Browse files
feat/migrate gdrive source connector (#3239)
### Description Migrate the google drive source connector over to the new v2 ingest framework and include a variety of improvements as part of the refactor: * The ID is no longer limited to a drive id but can also be the id of a subfolder within a drive or a file directly and each case is handled appropriately * More metadata is pulled in from google drive to enrich the partitioned elements downstream and now the modified date is being set to not reprocess if the ingest pipeline already has the file cached * timing information is set on the file created when downloaded based on the last modified data retrieved from google drive --------- Co-authored-by: ryannikolaidis <[email protected]> Co-authored-by: rbiseck3 <[email protected]>
1 parent e0f4374 commit 3f581e6

20 files changed

+6177
-1957
lines changed

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
## 0.14.9-dev0
1+
## 0.14.9-dev1
22

33
### Enhancements
44

test_unstructured_ingest/expected-structured-output/google-drive/117qrVqiCoR5EjYMsDHGdy3UMkEtKr9Q8-test-drive-doc.docx.json

Lines changed: 0 additions & 54 deletions
This file was deleted.

test_unstructured_ingest/expected-structured-output/google-drive/1SpQuE7jHz9nMt5hfQXsiok1SgIdRYX5o-fake.docx.json

Lines changed: 0 additions & 23 deletions
This file was deleted.

test_unstructured_ingest/expected-structured-output/google-drive/1cTKXAreuj-wYmL38nFnqKvz3X8UKcaMC-foo.txt.json

Lines changed: 0 additions & 22 deletions
This file was deleted.

0 commit comments

Comments
 (0)