|
| 1 | += google_drive_download |
| 2 | +// tag::single-source[] |
| 3 | +:type: processor |
| 4 | +:categories: ["AI"] |
| 5 | + |
| 6 | +component_type_dropdown::[] |
| 7 | + |
| 8 | +Downloads files from Google Drive that contain matching file IDs. |
| 9 | + |
| 10 | +Try out the <<example,example pipeline on this page>>, which downloads all files from your Google Drive. |
| 11 | + |
| 12 | +ifndef::env-cloud[] |
| 13 | +Introduced in version 4.53.0. |
| 14 | +endif::[] |
| 15 | + |
| 16 | +[tabs] |
| 17 | +====== |
| 18 | +Common:: |
| 19 | ++ |
| 20 | +-- |
| 21 | +
|
| 22 | +```yml |
| 23 | +# Common configuration fields, showing default values |
| 24 | +label: "" |
| 25 | +google_drive_download: |
| 26 | + credentials_json: "" # No default (optional) |
| 27 | + file_id: "" # No default (required) |
| 28 | + mime_type: "" # No default (required) |
| 29 | +``` |
| 30 | +
|
| 31 | +-- |
| 32 | +Advanced:: |
| 33 | ++ |
| 34 | +-- |
| 35 | +
|
| 36 | +```yml |
| 37 | +# All configuration fields, showing default values |
| 38 | +label: "" |
| 39 | +google_drive_download: |
| 40 | + credentials_json: "" # No default (optional) |
| 41 | + file_id: "" # No default (required) |
| 42 | + mime_type: "" # No default (required) |
| 43 | + export_mime_types: |
| 44 | + application/vnd.google-apps.document: text/markdown |
| 45 | + application/vnd.google-apps.drawing: image/png |
| 46 | + application/vnd.google-apps.presentation: application/pdf |
| 47 | + application/vnd.google-apps.script: application/vnd.google-apps.script+json |
| 48 | + application/vnd.google-apps.spreadsheet: text/csv |
| 49 | +``` |
| 50 | +
|
| 51 | +-- |
| 52 | +====== |
| 53 | + |
| 54 | +== Authentication |
| 55 | + |
| 56 | +By default, this processor uses https://cloud.google.com/docs/authentication/application-default-credentials[Google Application Default Credentials (ADC)^] to authenticate with Google APIs. |
| 57 | + |
| 58 | +To set up local ADC authentication, use the following `gcloud` commands: |
| 59 | + |
| 60 | +- Authenticate using Application Default Credentials and grant read-only access to your Google Drive. |
| 61 | ++ |
| 62 | +```bash |
| 63 | +gcloud auth application-default login --scopes='openid,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/drive.readonly' |
| 64 | +``` |
| 65 | +- Assign a quota project to the Application Default Credentials when using a user account. |
| 66 | ++ |
| 67 | +```bash |
| 68 | +gcloud auth application-default set-quota-project <project-id> |
| 69 | +``` |
| 70 | ++ |
| 71 | +Replace the `<project-id>` placeholder with your Google Cloud project ID |
| 72 | + |
| 73 | +To use a service account instead, create a JSON key for the account and add it to the <<credentials_json,`credentials_json`>> field. To access Google Drive files using a service account, either: |
| 74 | + |
| 75 | +- Explicitly share files with the service account's email account |
| 76 | +- Use https://support.google.com/a/answer/162106[domain-wide delegation^] to share all files within a Google Workspace |
| 77 | + |
| 78 | +== Fields |
| 79 | + |
| 80 | +=== `credentials_json` |
| 81 | + |
| 82 | +The JSON key for your service account (optional). If left empty, Application Default Credentials are used. For more details, see <<authentication, Authentication>>. |
| 83 | + |
| 84 | +include::components:partial$secret_warning.adoc[] |
| 85 | + |
| 86 | +*Type*: `string` |
| 87 | + |
| 88 | +*Default*: `""` |
| 89 | + |
| 90 | +=== `file_id` |
| 91 | + |
| 92 | +The ID of the file to download from Google Drive. This field supports xref:configuration:interpolation.adoc#bloblang-queries[interpolation functions]. |
| 93 | + |
| 94 | +*Type*: `string` |
| 95 | + |
| 96 | +*Default*: `""` |
| 97 | + |
| 98 | +=== `mime_type` |
| 99 | + |
| 100 | +The https://developers.google.com/workspace/drive/api/guides/mime-types[MIME type^] of the file for download. This field supports xref:configuration:interpolation.adoc#bloblang-queries[interpolation functions]. |
| 101 | + |
| 102 | +*Type*: `string` |
| 103 | + |
| 104 | +*Default*: `""` |
| 105 | + |
| 106 | +=== `export_mime_types` |
| 107 | + |
| 108 | +Maps Google Drive MIME types to https://developers.google.com/workspace/drive/api/guides/ref-export-formats[supported file export formats^]. The MIME type is the key, and the export format is the value. |
| 109 | + |
| 110 | +*Type*: `object` |
| 111 | + |
| 112 | +*Default*: `{"application/vnd.google-apps.document":"text/markdown","application/vnd.google-apps.drawing":"image/png","application/vnd.google-apps.presentation":"application/pdf","application/vnd.google-apps.script":"application/vnd.google-apps.script+json","application/vnd.google-apps.spreadsheet":"text/csv"}` |
| 113 | + |
| 114 | +```yml |
| 115 | +# Examples |
| 116 | + |
| 117 | +export_mime_types: |
| 118 | + application/vnd.google-apps.document: application/pdf |
| 119 | + application/vnd.google-apps.drawing: application/pdf |
| 120 | + application/vnd.google-apps.presentation: application/pdf |
| 121 | + application/vnd.google-apps.spreadsheet: application/pdf |
| 122 | + |
| 123 | +export_mime_types: |
| 124 | + application/vnd.google-apps.document: application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| 125 | + application/vnd.google-apps.drawing: image/svg+xml |
| 126 | + application/vnd.google-apps.presentation: application/vnd.openxmlformats-officedocument.presentationml.presentation |
| 127 | + application/vnd.google-apps.spreadsheet: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet |
| 128 | +``` |
| 129 | + |
| 130 | +== Example |
| 131 | + |
| 132 | +This example downloads all files from a Google Drive. |
| 133 | + |
| 134 | +```yaml |
| 135 | +input: |
| 136 | + stdin: {} |
| 137 | +pipeline: |
| 138 | + processors: |
| 139 | + - google_drive_search: |
| 140 | + query: "${!content().string()}" |
| 141 | + - mutation: 'meta path = this.name' |
| 142 | + - google_drive_download: |
| 143 | + file_id: "${!this.id}" |
| 144 | + mime_type: "${!this.mimeType}" |
| 145 | +output: |
| 146 | + file: |
| 147 | + path: "${!@path}" |
| 148 | + codec: all-bytes |
| 149 | +``` |
| 150 | + |
| 151 | + |
| 152 | +// end::single-source[] |
0 commit comments