Databricks Volumes and Delta Tables connectors: Add more how-to video links, and required permissions details (#465)

Paul-Cornell · web-flow · commit 3d5a31596ea9 · 2025-02-03T08:28:06.000-05:00
diff --git a/snippets/general-shared-text/databricks-delta-table-api-placeholders.mdx b/snippets/general-shared-text/databricks-delta-table-api-placeholders.mdx
@@ -2,7 +2,7 @@
 - `<server-hostname>` (_required_): The target Databricks cluster's or SQL warehouse's **Server Hostname** value.
 - `<http-path>` (_required_): The cluster's or SQL warehouse's **HTTP Path** value.
 - `<token>` (_required_ for PAT authentication): For Databricks personal access token (PAT) authentication, the target Databricks user's PAT value.
-- `<client-id>` and `<client-secret>` (_required_ for OAuth authentication): For Databricks OAuth machine-to-machine (M2M) authentication, the Databricks managed service principal's **UUID** (client ID or Application ID) and OAuth **Secret** (client secret) values. 
+- `<client-id>` and `<client-secret>` (_required_ for OAuth authentication): For Databricks OAuth machine-to-machine (M2M) authentication, the Databricks managed service principal's **UUID** (or **Client ID** or **Application ID**) and OAuth **Secret** (client secret) values. 
 - `<catalog>` (_required_): The name of the catalog in Unity Catalog for the target volume and table in the Databricks workspace.
 - `<database>`: The name of the database in Unity Catalog for the target volume and table. The default is `default` if not otherwise specified.
 - `<table-name>` (_required_): The name of the target table in Unity Catalog.
diff --git a/snippets/general-shared-text/databricks-delta-table-cli-api.mdx b/snippets/general-shared-text/databricks-delta-table-cli-api.mdx
@@ -13,7 +13,7 @@ The following environment variables:
 - `DATABRICKS_HOST` - The Databricks cluster's or SQL warehouse's **Server Hostname** value, represented by `--server-hostname` (CLI) or `server_hostname` (Python).
 - `DATABRICKS_HTTP_PATH` - The cluster's or SQL warehouse's **HTTP Path** value, represented by `--http-path` (CLI) or `http_path` (Python).
 - `DATABRICKS_TOKEN` - For Databricks personal access token authentication, the token's value, represented by `--token` (CLI) or `token` (Python).
-- `DATABRICKS_CLIENT_ID` - For Databricks managed service principal authenticaton, the service principal's **UUID** value, represented by `--client-id` (CLI) or `client_id` (Python).
+- `DATABRICKS_CLIENT_ID` - For Databricks managed service principal authenticaton, the service principal's **UUID** (or **Client ID** or **Application ID**) value, represented by `--client-id` (CLI) or `client_id` (Python).
 - `DATABRICKS_CLIENT_SECRET` - For Databricks managed service principal authenticaton, the service principal's OAuth **Secret** value, represented by `--client-secret` (CLI) or `client_secret` (Python).
 - `DATABRICKS_CATALOG` - The name of the catalog in Unity Catalog, represented by `--catalog` (CLI) or `catalog` (Python).
 - `DATABRICKS_DATABASE` - The name of the schema (database) inside of the catalog, represented by `--database` (CLI) or `database` (Python). The default is `default` if not otherwise specified.
diff --git a/snippets/general-shared-text/databricks-delta-table-platform.mdx b/snippets/general-shared-text/databricks-delta-table-platform.mdx
@@ -4,7 +4,7 @@ Fill in the following fields:
 - **Server Hostname** (_required_): The target Databricks cluster's or SQL warehouse's **Server Hostname** value.
 - **HTTP Path** (_required_): The cluster's or SQL warehouse's **HTTP Path** value.
 - **Token** (_required_ for PAT authentication): For Databricks personal access token (PAT) authentication, the target Databricks user's PAT value.
-- **UUID** and **OAuth Secret** (_required_ for OAuth authentication): For Databricks OAuth machine-to-machine (M2M) authentication, the Databricks managed service principal's **UUID** (client ID or Application ID) and OAuth **Secret** (client secret) values. 
+- **UUID** and **OAuth Secret** (_required_ for OAuth authentication): For Databricks OAuth machine-to-machine (M2M) authentication, the Databricks managed service principal's **UUID** (or **Client ID** or **Application ID**) and OAuth **Secret** (client secret) values. 
 - **Catalog** (_required_): The name of the catalog in Unity Catalog for the target volume and table in the Databricks workspace.
 - **Database**: The name of the database in Unity Catalog for the target volume and table. The default is `default` if not otherwise specified.
 - **Table Name** (_required_): The name of the target table in Unity Catalog.
diff --git a/snippets/general-shared-text/databricks-delta-table.mdx b/snippets/general-shared-text/databricks-delta-table.mdx
@@ -9,7 +9,7 @@
   - A SQL warehouse for [AWS](https://docs.databricks.com/compute/sql-warehouse/create.html), 
     [Azure](https://learn.microsoft.com/azure/databricks/compute/sql-warehouse/create), or 
     [GCP](https://docs.gcp.databricks.com/compute/sql-warehouse/create.html).
-  - A cluster for [AWS](https://docs.databricks.com/compute/use-compute.html), 
+  - An all-purpose cluster for [AWS](https://docs.databricks.com/compute/use-compute.html), 
     [Azure](https://learn.microsoft.com/azure/databricks/compute/use-compute), or 
     [GCP](https://docs.gcp.databricks.com/compute/use-compute.html).
 
@@ -102,7 +102,7 @@
 
   - A Databricks managed service principal. 
     This service principal must have the appropriate access permissions to the catalog, schema, table, volume, and cluster or SQL warehouse.
-  - The service principal's **UUID** value.
+  - The service principal's **UUID** (or **Client ID** or **Application ID**) value.
   - The OAuth **Secret** value for the service principal. 
 
   To get this information, see Steps 1-3 of the instructions for [AWS](https://docs.databricks.com/dev-tools/auth/oauth-m2m.html), 
@@ -112,4 +112,61 @@
   <Note>
       For Azure Databricks, this connector only supports Databricks managed service principals. 
       Microsoft Entra ID managed service principals are not supported.
-  </Note>
+  </Note>
+
+  The following video shows how to create a Databricks managed service principal:
+
+  <iframe
+  width="560"
+  height="315"
+  src="https://www.youtube.com/embed/wBmqv5DaA1E"
+  title="YouTube video player"
+  frameborder="0"
+  allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
+  allowfullscreen
+  ></iframe>
+
+- The Databricks workspace user or Databricks managed service principal must have the following _minimum_ set of permissions and privileges to write to an 
+  existing volume or table in Unity Catalog:
+
+  - To use an all-purpose cluster for access, `Can Restart` permission on that cluster. Learn how to check and set cluster permissions for 
+    [AWS](https://docs.databricks.com/compute/clusters-manage.html#compute-permissions), 
+    [Azure](https://learn.microsoft.com/azure/databricks/compute/clusters-manage#cluster-level-permissions), or 
+    [GCP](https://docs.gcp.databricks.com/compute/clusters-manage.html#compute-permissions).
+  - To use a SQL warehouse for access, `Can use` permission on that SQL warehouse. Learn how to check and set SQL warehouse permissions for 
+    [AWS](https://docs.databricks.com/compute/sql-warehouse/create.html#manage-a-sql-warehouse), 
+    [Azure](https://learn.microsoft.com/azure/databricks/compute/sql-warehouse/create#manage), or 
+    [GCP](https://docs.gcp.databricks.com/compute/sql-warehouse/create.html#manage-a-sql-warehouse).
+  - To access a Unity Catalog volume, the following privileges:
+
+    - `USE CATALOG` on the volume's parent catalog in Unity Catalog.
+    - `USE SCHEMA` on the volume's parent schema in Unity Catalog.
+    - `READ VOLUME` and `WRITE VOLUME` on the volume.
+
+    Learn how to check and set Unity Catalog privileges for 
+    [AWS](https://docs.databricks.com/data-governance/unity-catalog/manage-privileges/index.html#show-grant-and-revoke-privileges), 
+    [Azure](https://learn.microsoft.com/azure/databricks/data-governance/unity-catalog/manage-privileges/#grant), or 
+    [GCP](https://docs.gcp.databricks.com/data-governance/unity-catalog/manage-privileges/index.html#show-grant-and-revoke-privileges).
+
+    The following videos shows how to grant a Databricks managed service principal privileges to a Unity Catalog volume:
+    
+    <iframe
+    width="560"
+    height="315"
+    src="https://www.youtube.com/embed/DykQRxgh2aQ"
+    title="YouTube video player"
+    frameborder="0"
+    allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
+    allowfullscreen
+    ></iframe>
+
+  - To access a Unity Catalog table, the following privileges:
+
+    - `USE CATALOG` on the table's parent catalog in Unity Catalog.
+    - `USE SCHEMA` on the tables's parent schema in Unity Catalog.
+    - `MODIFY` and `SELECT` on the table.
+
+    Learn how to check and set Unity Catalog privileges for 
+    [AWS](https://docs.databricks.com/data-governance/unity-catalog/manage-privileges/index.html#show-grant-and-revoke-privileges), 
+    [Azure](https://learn.microsoft.com/azure/databricks/data-governance/unity-catalog/manage-privileges/#grant), or 
+    [GCP](https://docs.gcp.databricks.com/data-governance/unity-catalog/manage-privileges/index.html#show-grant-and-revoke-privileges).
diff --git a/snippets/general-shared-text/databricks-volumes-api-placeholders.mdx b/snippets/general-shared-text/databricks-volumes-api-placeholders.mdx
@@ -1,21 +1,8 @@
 - `<name>` (_required_) - A unique name for this connector.
 - `<host>` (_required_) - The Databricks workspace host URL.
-- `<client-id>` (_required_) - The application ID value for the Databricks-managed service principal that has access to the volume.
-- `<client-secret>` (_required_) - The associated OAuth secret value for the Databricks-managed service principal that has access to the volume.
+- `<client-id>` (_required_) - The **Client ID** (or **UUID** or **Application ID**) value for the Databricks managed service principal that has the appropriate privileges to the volume.
+- `<client-secret>` (_required_) - The associated OAuth **Secret** value for the Databricks managed service principal that has the appropriate privileges to the volume.
 - `<catalog>` (_required_) - The name of the catalog to use.
 - `<schema>` - The name of the associated schema. If not specified, `default` is used.
 - `<volume>` (_required_) - The name of the associated volume.
-- `<volume-path>` - Any optional path to access within the volume.
-
-To learn how to create a Databricks-managed service principal, get its application ID, and generate an associated OAuth secret, 
-see the documentation for 
-[AWS](https://docs.databricks.com/dev-tools/auth/oauth-m2m.html), 
-[Azure](https://learn.microsoft.com/databricks/dev-tools/auth/oauth-m2m), 
-or [GCP](https://docs.gcp.databricks.com/dev-tools/auth/oauth-m2m.html).
-
-For Azure, only Databricks-managed service principals are supported. Microsoft Entra ID-managed service principals are not supported.
-
-To learn how to grant a Databricks-managed service principal access to a volume, see the documentation for 
-[AWS](https://docs.databricks.com/volumes/utility-commands.html#change-permissions-on-a-volume), 
-[Azure](https://learn.microsoft.com/azure/databricks/volumes/utility-commands#change-permissions-on-a-volume), 
-or [GCP](https://docs.gcp.databricks.com/volumes/utility-commands.html#change-permissions-on-a-volume).
+- `<volume-path>` - Any optional path to access within the volume.
diff --git a/snippets/general-shared-text/databricks-volumes-platform.mdx b/snippets/general-shared-text/databricks-volumes-platform.mdx
@@ -6,21 +6,6 @@ Fill in the following fields:
 - **Schema** : The name of the associated schema. If not specified, **default** is used.
 - **Volume** (_required_): The name of the associated volume.
 - **Volume Path** : Any optional path to access within the volume.
-- **Client Secret** (_required_): The associated OAuth secret value for the Databricks-managed service principal that has access to the volume.
-- **Client ID** (_required_): The application ID value for the Databricks-managed service principal that has access to the volume.
-
-To learn how to create a Databricks-managed service principal, get its application ID, and generate an associated OAuth secret, 
-see the documentation for 
-[AWS](https://docs.databricks.com/dev-tools/auth/oauth-m2m.html), 
-[Azure](https://learn.microsoft.com/databricks/dev-tools/auth/oauth-m2m), 
-or [GCP](https://docs.gcp.databricks.com/dev-tools/auth/oauth-m2m.html).
-
-For Azure, only Databricks-managed service principals are supported. Microsoft Entra ID-managed service principals are not supported.
-
-To learn how to grant a Databricks-managed service principal access to a volume, see the documentation for 
-[AWS](https://docs.databricks.com/volumes/utility-commands.html#change-permissions-on-a-volume), 
-[Azure](https://learn.microsoft.com/azure/databricks/volumes/utility-commands#change-permissions-on-a-volume), 
-or [GCP](https://docs.gcp.databricks.com/volumes/utility-commands.html#change-permissions-on-a-volume).
-
-
+- **Client Secret** (_required_): The associated OAuth **Secret** value for the Databricks managed service principal that has the appropriate privileges to the volume.
+- **Client ID** (_required_): The **Client ID** (or **UUID** or **Application ID**) value for the Databricks managed service principal that has appropriate privileges to the volume.
 
diff --git a/snippets/general-shared-text/databricks-volumes.mdx b/snippets/general-shared-text/databricks-volumes.mdx
@@ -10,8 +10,8 @@ allowfullscreen
 
 The preceding video shows how to use Databricks personal access tokens (PATs), which are supported only for [Unstructured Ingest](/ingestion/overview).
 
-To learn how to use Databricks-managed service principals, which are supported by both the [Unstructured Platform](/platform/overview) and Unstructured Ingest, 
-see the additional videos later on this page.
+To learn how to use Databricks managed service principals, which are supported by both the [Unstructured Platform](/platform/overview) and Unstructured Ingest, 
+see the additional video later on this page.
 
 - The Databricks workspace URL. Get the workspace URL for 
   [AWS](https://docs.databricks.com/workspace/workspace-details.html#workspace-instance-names-urls-and-ids), 
@@ -29,7 +29,7 @@ see the additional videos later on this page.
   [Azure](https://learn.microsoft.com/azure/databricks/dev-tools/auth/), 
   or [GCP](https://docs.gcp.databricks.com/dev-tools/auth/index.html).
 
-  The following videos show how to create a Databricks-managed service principal and then grant it access to a Databricks volume:
+  The following video shows how to create a Databricks managed service principal:
 
   <iframe
   width="560"
@@ -41,20 +41,9 @@ see the additional videos later on this page.
   allowfullscreen
   ></iframe>
 
-  <iframe
-  width="560"
-  height="315"
-  src="https://www.youtube.com/embed/DykQRxgh2aQ"
-  title="YouTube video player"
-  frameborder="0"
-  allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
-  allowfullscreen
-  ></iframe>
-
-  For the [Unstructured Platform](/platform/overview), only the following Databricks authentication type is supported:
-  
-  - For OAuth machine-to-machine (M2M) authentication (AWS, Azure, and GCP): The client ID and OAuth secret values for the corresponding service principal.
-    Note that for Azure, only Databricks-managed service principals are supported. Microsoft Entra ID-managed service principals are not supported.
+  For the [Unstructured Platform](/platform/overview), only Databricks OAuth machine-to-machine (M2M) authentication is supported for AWS, Azure, and GCP. 
+  You will need the the **Client ID** (or **UUID** or **Application** ID) and OAuth **Secret** (client secret) values for the corresponding service principal. 
+  Note that for Azure, only Databricks managed service principals are supported. Microsoft Entra ID managed service principals are not supported.
   
   For [Unstructured Ingest](/ingestion/overview), the following Databricks authentication types are supported:
 
@@ -69,10 +58,37 @@ see the additional videos later on this page.
   - For Google Cloud Platform credentials authentication (GCP only): The local path to the corresponding Google Cloud service account's credentials file.
   - For Google Cloud Platform ID authentication (GCP only): The Google Cloud service account's email address.
 
-- The Databricks catalog name for the volume. Get the catalog name for [AWS](https://docs.databricks.com/catalogs/manage-catalog.html), [Azure](https://learn.microsoft.com/azure/databricks/catalogs/manage-catalog), or [GCP](https://docs.gcp.databricks.com/catalogs/manage-catalog.html).
-- The Databricks schema name for the volume. Get the schema name for [AWS](https://docs.databricks.com/schemas/manage-schema.html), [Azure](https://learn.microsoft.com/azure/databricks/schemas/manage-schema), or [GCP](https://docs.gcp.databricks.com/schemas/manage-schema.html).
-- The Databricks volume name, and optionally any path in that volume that you want to access directly. Get the volume information for [AWS](https://docs.databricks.com/files/volumes.html), [Azure](https://learn.microsoft.com/azure/databricks/files/volumes), or [GCP](https://docs.gcp.databricks.com/files/volumes.html).
-- Make sure that the target user or service principal has access to the target volume. To learn more, see the documentation for 
-  [AWS](https://docs.databricks.com/volumes/utility-commands.html#change-permissions-on-a-volume), 
-  [Azure](https://learn.microsoft.com/azure/databricks/volumes/utility-commands#change-permissions-on-a-volume), 
-  or [GCP](https://docs.gcp.databricks.com/volumes/utility-commands.html#change-permissions-on-a-volume).
+- The name of the parent catalog in Unity Catalog for 
+  [AWS](https://docs.databricks.com/catalogs/create-catalog.html), 
+  [Azure](https://learn.microsoft.com/azure/databricks/catalogs/create-catalog), or 
+  [GCP](https://docs.gcp.databricks.com/catalogs/create-catalog.html) for the volume.
+- The name of the parent schema in Unity Catalog for 
+  [AWS](https://docs.databricks.com/schemas/create-schema.html), 
+  [Azure](https://learn.microsoft.com/azure/databricks/schemas/create-schema), or 
+  [GCP](https://docs.gcp.databricks.com/schemas/create-schema.html) for the volume.
+- The name of the volume in Unity Catalog for [AWS](https://docs.databricks.com/tables/managed.html), 
+  [Azure](https://learn.microsoft.com/azure/databricks/tables/managed), or 
+  [GCP](https://docs.gcp.databricks.com/tables/managed.html), and optionally any path in that volume that you want to access directly, beginning with the volume's root.
+- The Databricks workspace user or service principal must have the following _minimum_ set of privileges to read from or write to the 
+  existing volume in Unity Catalog:
+
+  - `USE CATALOG` on the volume's parent catalog in Unity Catalog.
+  - `USE SCHEMA` on the volume's parent schema in Unity Catalog.
+  - `READ VOLUME` and `WRITE VOLUME` on the volume.
+
+  Learn how to check and set Unity Catalog privileges for 
+  [AWS](https://docs.databricks.com/data-governance/unity-catalog/manage-privileges/index.html#show-grant-and-revoke-privileges), 
+  [Azure](https://learn.microsoft.com/azure/databricks/data-governance/unity-catalog/manage-privileges/#grant), or 
+  [GCP](https://docs.gcp.databricks.com/data-governance/unity-catalog/manage-privileges/index.html#show-grant-and-revoke-privileges).
+
+  The following videos shows how to grant a Databricks managed service principal privileges to a Unity Catalog volume:
+    
+  <iframe
+  width="560"
+  height="315"
+  src="https://www.youtube.com/embed/DykQRxgh2aQ"
+  title="YouTube video player"
+  frameborder="0"
+  allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
+  allowfullscreen
+  ></iframe>
