Skip to content

Commit 16f8500

Browse files
authored
[Hold] Platform: Google Drive source connector (#383)
1 parent 186d36d commit 16f8500

File tree

6 files changed

+28
-9
lines changed

6 files changed

+28
-9
lines changed

mint.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -477,6 +477,7 @@
477477
"platform/sources/dropbox",
478478
"platform/sources/elasticsearch",
479479
"platform/sources/google-cloud",
480+
"platform/sources/google-drive",
480481
"platform/sources/mongodb",
481482
"platform/sources/onedrive",
482483
"platform/sources/outlook",

platform/connectors.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ The Unstructured Platform supports connecting to the following source and destin
1818
- [Dropbox](/platform/sources/dropbox)
1919
- [Elasticsearch](/platform/sources/elasticsearch)
2020
- [Google Cloud Storage](/platform/sources/google-cloud)
21+
- [Google Drive](/platform/sources/google-drive)
2122
- [MongoDB](/platform/sources/mongodb)
2223
- [OneDrive](/platform/sources/onedrive)
2324
- [Outlook](/platform/sources/outlook)

platform/sources/overview.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ To create a source connector:
2525
- [Dropbox](/platform/sources/dropbox)
2626
- [Elasticsearch](/platform/sources/elasticsearch)
2727
- [Google Cloud Storage](/platform/sources/google-cloud)
28+
- [Google Drive](/platform/sources/google-drive)
2829
- [MongoDB](/platform/sources/mongodb)
2930
- [OneDrive](/platform/sources/onedrive)
3031
- [Outlook](/platform/sources/outlook)

snippets/general-shared-text/google-drive-cli-api.mdx

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,5 +11,7 @@ import AdditionalIngestDependencies from '/snippets/general-shared-text/ingest-d
1111
The following environment variables:
1212

1313
- `GOOGLE_DRIVE_FOLDER_ID` - The folder ID, represented by `--drive-id` (CLI) or `drive_id` (Python).
14-
- `GCP_SERVICE_ACCOUNT_KEY_FILEPATH` - The path to the `credentials.json` key file, represented by `--service-account-key-path` (CLI) or `service_account_key_path` (Python), or
15-
- `GCP_SERVICE_ACCOUNT_KEY_STRING` - The contents of the `credentials.json` key file as a string, represented by `--service-account-key` (CLI) or `service_account_key` (Python).
14+
- One of the following:
15+
16+
- `GCP_SERVICE_ACCOUNT_KEY_FILEPATH` - The path to the `credentials.json` key file, represented by `--service-account-key-path` (CLI) or `service_account_key_path` (Python).
17+
- `GCP_SERVICE_ACCOUNT_KEY_STRING` - The contents of the `credentials.json` key file as a string, represented by `--service-account-key` (CLI) or `service_account_key` (Python).
Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
Fill in the following fields:
22

33
- **Name** (_required_): A unique name for this connector.
4-
- **Drive ID** (_required_): The folder ID.
5-
- **Service Account Key** (_required_): The `credentials.json` key file's contents in JSON format.
6-
- **Extension**: Any file extensions to be included in the ingestion process (such as `.jpg`, `.docx`, and so on), if filtering is needed.
7-
- **Recursive**: Check this box to recursively access files from subfolders within the drive.
4+
- **Drive ID** (_required_): The target folder's ID.
5+
- **Extensions**: A comma-separated list of any file extensions to be included in the ingestion process (such as `.jpg,.pdf`), if filtering is needed.
6+
The default is to include all files, if not otherwise specified.
7+
- **Recursive**: Check this box to also access files from all subfolders within the folder.
8+
- **Account Key** (_required_): The contents of the `credentials.json` key file for the target service account. These contents must be expressed as a single-line string without line breaks.

snippets/general-shared-text/google-drive.mdx

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,22 @@
1010
allowfullscreen
1111
></iframe>
1212

13-
2. Note the local path to the `credentials.json` key file. Or, copy the key file's contents into a compatible string&mdash;including properly escaped quotes&mdash;as required.
14-
To make converting a JSON object into a compatible string easier, you can search the Internet by using a search phrase such as "tools for converting a JSON object into a string."
15-
Before using any tool, check to make sure that the tool does not share the key file's contents with anyone you do not trust.
13+
2. To ensure maximum compatibility across Unstructured service offerings, you should give the service account key information to Unstructured as
14+
a single-line string that contains the contents of the downloaded service account key file (and not the service account key file itself).
15+
To print this single-line string without line breaks, suitable for copying, you can run one of the following commands from your Terminal or Command Prompt.
16+
In this command, replace `<path-to-downloaded-key-file>` with the path to the `credentials.json` key file that you downloaded by following the preceding instructions.
17+
18+
- For macOS or Linux:
19+
20+
```text
21+
tr -d '\n' < <path-to-downloaded-key-file>
22+
```
23+
24+
- For Windows:
25+
26+
```text
27+
(Get-Content -Path "<path-to-downloaded-key-file>" -Raw).Replace("`r`n", "").Replace("`n", "")
28+
```
1629

1730
3. Give the service account's email address access to the Google Drive folder. [Learn more](https://www.googlecloudcommunity.com/gc/Workspace-Q-A/Can-i-give-access-to-document-of-google-drive-to-service-account/m-p/530106).
1831

0 commit comments

Comments
 (0)