Skip to content

Commit 623671b

Browse files
committed
Add overview and other PM feedback
1 parent 3ade417 commit 623671b

File tree

2 files changed

+38
-16
lines changed

2 files changed

+38
-16
lines changed

src/connections/storage/databricks-delta-lake/databricks-delta-lake-aws.md

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,24 @@ title: Databricks Delta Lake Destination (AWS Setup)
33
beta: true
44
---
55

6-
With the Databricks Destination, you can ingest event data from Segment into the bronze layer of your Databricks Delta Lake.
6+
With the Databricks Delta Lake Destination, you can ingest event data from Segment into the bronze layer of your Databricks Delta Lake.
77

8-
This page will help you use the Databricks Destination to sync Segment events into your Databricks Delta Lake built on AWS.
8+
This page will help you use the Databricks Destination to sync Segment events into your Databricks Delta Lake built on S3.
99

10-
> info "Databricks Delta Lake Destination in beta"
11-
> The Databricks Delta Lake Destination is in beta, and Segment is actively working on this feature. Some functionality may change before it becomes generally available. [Contact Segment](https://segment.com/help/contact/){:target="_blank"} with any feedback or questions.
10+
> info "Databricks Delta Lake Destination in public beta"
11+
> The Databricks Delta Lake Destination is in public beta, and Segment is actively working on this integration. [Contact Segment](https://segment.com/help/contact/){:target="_blank"} with any feedback or questions.
12+
13+
## Overview
14+
15+
Before getting started, use the overview below to get up to familiarize yourself with Segment's Databricks Delta Lake Destination.
16+
17+
1. Segment writes directly to your Delta Lake in the cloud storage (S3)
18+
- Segment manages the creation and evolution of Delta tables.
19+
- Segment uses IAM role assumption to write Delta to AWS S3.
20+
2. Segment supports both OAuth and personal access tokens (PAT) for API authentication.
21+
3. Segment creates and updates the table's metadeta in Unity Catalog by running queries on a small, single node Databricks SQL warehouse in your environment.
22+
4. If a table already exists and no new columns are introduced, Segment appends data to the table (no SQL required).
23+
5. For new data types/columns, Segment reads the current schema for the table from the Unity Catalog and uses the SQL warehouse to update the schema accordingly.
1224

1325
## Prerequisites
1426

@@ -32,9 +44,7 @@ As you set up Databricks, keep the following key terms in mind.
3244
- **Target Unity Catalog**: The catalog where Segment lands your data.
3345
- **Workspace Admin Token** (*PAT only*): The access token you'll generate for your Databricks workspace admin.
3446

35-
## Setup
36-
37-
Use the following nine steps to set up your Databricks Delta Lake destination with AWS.
47+
## Setup for Databricks Delta Lake (S3)
3848

3949
### Step 1: Find your Databricks Workspace URL
4050

@@ -149,7 +159,8 @@ This catalog is the target catalog where Segment lands your schemas/tables.
149159
### Step 9: Setup the Databricks Delta Lake destination in Segment
150160

151161
This step links a Segment events source to your Databricks workspace/catalog.
152-
1. Navigate to `https://app.segment.com/<WORKSPACE_SLUG>/destinations/catalog/databricks-delta-lake`.
162+
1. From the Segment app, navigate to **Connections > Catalog**, then click **Destinations**.
163+
2. Search for and select the "Databricks Delta Lake" destination.
153164
2. Click **Add Destination**, select a source, then click **Next**.
154165
3. Enter the name for your destination, then click **Create destination**.
155166
4. Enter connection settings for the destination.

src/connections/storage/databricks-delta-lake/databricks-delta-lake-azure.md

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,25 @@ title: Databricks Delta Lake Destination (Azure Setup)
33
beta: true
44
---
55

6-
With the Databricks Destination, you can ingest event data from Segment into the bronze layer of your Databricks Delta Lake.
6+
With the Databricks Delta Lake Destination, you can ingest event data from Segment into the bronze layer of your Databricks Delta Lake.
77

88
This page will help you use the Databricks Destination to sync Segment events into your Databricks Delta Lake built on Azure (ADLS Gen 2).
99

1010

11-
> info "Databricks Delta Lake Destination in beta"
12-
> The Databricks Delta Lake Destination is in beta, and Segment is actively working on this feature. Some functionality may change before it becomes generally available. [Contact Segment](https://segment.com/help/contact/){:target="_blank"} with any feedback or questions.
11+
> info "Databricks Delta Lake Destination in public beta"
12+
> The Databricks Delta Lake Destination is in public beta, and Segment is actively working on this integration. [Contact Segment](https://segment.com/help/contact/){:target="_blank"} with any feedback or questions.
13+
14+
## Overview
15+
16+
Before getting started, use the overview below to get up to familiarize yourself with Segment's Databricks Delta Lake Destination.
17+
18+
1. Segment writes directly to your Delta Lake in the cloud storage (Azure)
19+
- Segment manages the creation and evolution of Delta tables.
20+
- Segment uses a cross-tenant service principal to write Delta to ADLS Gen2.
21+
2. Segment supports both OAuth and personal access tokens (PAT) for API authentication.
22+
3. Segment creates and updates the table's metadeta in Unity Catalog by running queries on a small, single node Databricks SQL warehouse in your environment.
23+
4. If a table already exists and no new columns are introduced, Segment appends data to the table (no SQL required).
24+
5. For new data types/columns, Segment reads the current schema for the table from the Unity Catalog and uses the SQL warehouse to update the schema accordingly.
1325

1426
## Prerequisites
1527

@@ -27,9 +39,7 @@ As you set up Databricks, keep the following key terms in mind.
2739
- **Databricks Workspace URL**: The base URL for your Databricks workspace.
2840
- **Target Unity Catalog**: The catalog where Segment lands your data.
2941

30-
## Set up Databricks with Azure
31-
32-
Use the following eight steps to setup your Databricks Delta Lake destination with Azure.
42+
## Set up Databricks Delta Lake (Azure)
3343

3444
### Step 1: Find your Databricks Workspace URL
3545

@@ -116,7 +126,8 @@ This catalog is the target catalog where Segment lands your schemas/tables.
116126
### Step 8: Setup the Databricks Delta Lake destination in Segment
117127

118128
This step links a Segment source to your Databricks workspace/catalog.
119-
1. Navigate to `https://app.segment.com/<WORKSPACE_SLUG>/destinations/catalog/databricks-delta-lake`.
129+
1. From the Segment app, navigate to **Connections > Catalog**, then click **Destinations**.
130+
2. Search for and select the "Databricks Delta Lake" destination.
120131
2. Click **Add Destination**, select a source, then click **Next**.
121132
3. Enter the name for your destination, then click **Create destination**.
122-
4. Enter the connection settings using the values noted above (leave the service principal fields blank).
133+
4. Enter the connection settings using the values noted above (leave the Service Principal fields blank).

0 commit comments

Comments
 (0)