Google Cloud Storage source and destinations connectors: bucket access and key file instructions, add to Platform (#326)

Author: Paul-Cornell

mint.json

@@ -449,6 +449,7 @@
               "pages": [
                 "platform/sources/overview",
                 "platform/sources/azure-blob-storage",
+                "platform/sources/google-cloud",
                 "platform/sources/s3",
                 "platform/sources/sharepoint"
               ]
@@ -460,6 +461,7 @@
               "platform/destinations/astradb",
               "platform/destinations/azure-cognitive-search",
               "platform/destinations/delta-table",
+              "platform/destinations/google-cloud",
               "platform/destinations/milvus",
               "platform/destinations/mongodb",
               "platform/destinations/pinecone",

platform/connectors.mdx

@@ -12,6 +12,7 @@ The Unstructured Platform supports connecting to the following source and destin
 ## Sources
 
 - [Azure](/platform/sources/azure-blob-storage)
+- [Google Cloud Storage](/platform/sources/google-cloud)
 - [S3](/platform/sources/s3)
 - [SharePoint](/platform/sources/sharepoint)
 
@@ -25,6 +26,7 @@ If your source is not listed here, you might still be able to connect Unstructur
 - [Astra DB](/platform/destinations/astradb)
 - [Azure Cognitive Search](/platform/destinations/azure-cognitive-search)
 - [Delta Table](/platform/destinations/delta-table)
+- [Google Cloud Storage](/platform/destinations/google-cloud)
 - [Milvus](/platform/destinations/milvus)
 - [MongoDB](/platform/destinations/mongodb)
 - [Pinecone](/platform/destinations/pinecone)

platform/destinations/google-cloud-storage.mdx

@@ -0,0 +1,26 @@
+---
+title: Google Cloud Storage
+---
+
+Send processed data from Unstructured to Google Cloud Storage.
+
+You'll need:
+
+import GCSPrerequisites from '/snippets/general-shared-text/gcs.mdx';
+
+<GCSPrerequisites />
+
+To create the destination connector:
+
+1. On the sidebar, click **Connectors**.
+2. Click **Destinations**.
+3. Click **Add new**.
+4. Give the connector some unique **Name**.
+5. In the **Provider** area, click **Google GCS**.
+6. Click **Continue**. 
+7. Follow the on-screen instructions to fill in the fields as described later on this page.
+8. Click **Save and Test**.
+
+import GCSFields from '/snippets/general-shared-text/gcs-platform.mdx';
+
+<GCSFields />

platform/destinations/google-cloud.mdx

@@ -20,6 +20,7 @@ To create a destination connector:
    - [Astra DB](/platform/destinations/astradb)
    - [Azure Cognitive Search](/platform/destinations/azure-cognitive-search)
    - [Delta Table](/platform/destinations/delta-table)
+   - [Google Cloud Storage](/platform/destinations/google-cloud)
    - [Milvus](/platform/destinations/milvus)
    - [MongoDB](/platform/destinations/mongodb)
    - [Pinecone](/platform/destinations/pinecone)

platform/destinations/overview.mdx

@@ -16,7 +16,7 @@ To create the source connector:
 2. Click **Sources**.
 3. Click **Add new**.
 4. Give the connector some unique **Name**.
-5. In the **Provider** area, click **Google Cloud Storage**.
+5. In the **Provider** area, click **Google GCS**.
 6. Click **Continue**. 
 7. Follow the on-screen instructions to fill in the fields as described later on this page.
 8. Click **Save and Test**.

platform/sources/google-cloud.mdx

@@ -19,6 +19,7 @@ To create a source connector:
 7. Fill in the fields according to your connector type. To learn how, click your connector type in the following list:
 
    - [Azure](/platform/sources/azure-blob-storage)
+   - [Google Cloud Storage](/platform/sources/google-cloud)
    - [S3](/platform/sources/s3)
    - [SharePoint](/platform/source/sharepoint)
 

platform/sources/overview.mdx

@@ -1,7 +1,6 @@
 Fill in the following fields:
 
 - **Name** (_required_): A unique name for this connector.
-- **Remote URL** (_required_): The `gs://` URL for the Google Cloud Storage bucket and path.
-- **Service Account Key** (_required_): The JSON content of a Google Cloud service account key that has the necessary permissions to access the specified bucket. The service account key must have at least the **Storage Object Viewer** role to ensure proper access permissions. [Create a service account key](https://cloud.google.com/iam/docs/keys-create-delete#creating). [Assign a role](https://cloud.google.com/storage/docs/access-control/iam).
-- **Uncompress archives**: Check this box if the files to be ingested are compressed and require uncompression.
-- **Recursive processing**: Check this box to ingest data recursively from subfolders within the bucket.
+- **Bucket URI** (_required_): The URI for the Google Cloud Storage bucket and any target folder path within the bucket. This URI takes the format `gs://<bucket-name>[/folder-name]`.
+- **Service Account Key** (_required_): The contents of a service account key file, expressed as a single string without line breaks, for a Google Cloud service account that has the required access permissions to the bucket.
+- **Recursive**: Check this box to ingest data recursively from any subfolders, starting from the path specified by **Bucket URI**.

snippets/general-shared-text/gcs-platform.mdx

@@ -1,6 +1,30 @@
 The Google Cloud Storage prerequisites:
 
-- A Google Cloud Storage bucket URL, beginning with `gs://`.
-- A Google Cloud service account key for Google Cloud Storage. The service account key must have at least the **Storage Object Viewer** role to ensure proper access permissions. [Create a service account key](https://cloud.google.com/iam/docs/keys-create-delete#creating). [Assign a role](https://cloud.google.com/storage/docs/access-control/iam).
+- A Google Cloud service account. [Create a service account](https://cloud.google.com/iam/docs/service-accounts-create#console).
+- A service account key for the service account. See [Create a service account key](https://cloud.google.com/iam/docs/keys-create-delete#creating) in
+  [Create and delete service account keys](https://cloud.google.com/iam/docs/keys-create-delete). 
 
-[Learn more](https://cloud.google.com/storage/docs).
+  To ensure maximum compatibility across Unstructured service offerings, you should give the service account key information to Unstructured as 
+  a single-line string that contains the contents of the downloaded service account key file (and not the service account key file itself). 
+  To print this single-line string without line breaks, suitable for copying, you can run one of the following commands from your Terminal or Command Prompt. 
+  In this command, replace `<path-to-downloaded-key-file>` with the path to the service account key file that you downloaded by following the preceding instructions.
+
+  - For macOS or Linux: 
+    ```text
+    tr -d '\n' < <path-to-downloaded-key-file>
+    ```
+  - For Windows:
+    ```text
+    (Get-Content -Path "<path-to-downloaded-key-file>" -Raw).Replace("`r`n", "").Replace("`n", "")
+    ```
+
+- The URI for a Google Cloud Storage bucket. This URI consists of the target bucket name, plus any target folder within the bucket, expressed as `gs://<bucket-name>[/folder-name]`. [Create a bucket](https://cloud.google.com/storage/docs/creating-buckets#console).
+
+  This bucket must have, at minimum, one of the following roles applied to the target Google Cloud service account:
+
+  - `Storage Object Viewer` for bucket read access.
+  - `Storage Object Creator` for bucket write access.
+  - The `Storage Object Admin` role provides read and write access, plus access to additional bucket operations.
+
+  To apply one of these roles to a service account for a bucket, see [Add a principal to a bucket-level policy](https://cloud.google.com/storage/docs/access-control/using-iam-permissions#bucket-add) 
+  in [Set and manage IAM policies on buckets](https://cloud.google.com/storage/docs/access-control/using-iam-permissions).