Skip to content

Commit 067a1e7

Browse files
authored
add databricks privatelink docs (#2058)
1 parent a94adb8 commit 067a1e7

File tree

9 files changed

+82
-7
lines changed

9 files changed

+82
-7
lines changed

docs/cloud/integrations/security-and-connectivity/aws-privatelink-integration.mdx

Lines changed: 72 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ In order to set up a PrivateLink connection with Snowflake, please follow the st
4646
3. **Provide Elementary with the configuration obtained in the previous step.**
4747
1. Elementary will then setup the required infrastructure to connect to Snowflake via PrivateLink.
4848
4. **Add a Snowflake environment in Elementary**
49-
1. Follow the instructions [here](https://docs.elementary-data.com/cloud/onboarding/connect-data-warehouse) to set up a Snowflake environment in Elementary.
49+
1. Follow the instructions [here](/cloud/integrations/dwh/snowflake) to set up a Snowflake environment in Elementary.
5050
1. When supplying the account, use `<account_identifier>.privatelink` , where the account identifier is the result of the following query:
5151

5252
```sql
@@ -58,6 +58,77 @@ In order to set up a PrivateLink connection with Snowflake, please follow the st
5858
* 10.0.2.x
5959
* 10.0.3.x
6060

61+
### Databricks
62+
63+
Databricks has support for connecting to AWS-hosted Databricks workspaces via PrivateLink. This setup is entirely managed by Databricks, so Elementary connects with an endpoint service hosted on Databrick's AWS account for this purpose.
64+
65+
**Note**:
66+
1. You must be a Databricks account admin to perform this setup.
67+
2. Your databricks workspace must be deployed on a customer-managed VPC. PrivateLink is not supported with Databricks-managed VPCs.
68+
69+
In order to set up a PrivateLink connection with Databricks, please follow the steps below:
70+
71+
1. **Please provide Elementary with the following details:**
72+
* Your databricks workspace URL.
73+
* Your AWS account ID.
74+
* Your AWS region.
75+
76+
Elementary will then provide you with a VPC Endpoint ID that will be used in the next step.
77+
78+
2. **Register your VPC endpoint**
79+
80+
In the account management portal (not your workspace), go to Security -> Networking -> VPC Endpoints,
81+
then click on the "Register VPC Endpoint" button.
82+
83+
You should fill in:
84+
1. A name for the VPC endpoint - e.g. "Elementary".
85+
2. Your AWS region.
86+
3. The VPC Endpoint ID provided to you by Elementary.
87+
88+
<img
89+
src="https://res.cloudinary.com/dgpojk42n/image/upload/v1763576281/databricks_privatelink_02_register_vpc_endpoint_tdzer3.png"
90+
alt="Register VPC Endpoint"
91+
/>
92+
93+
3. **Configure a private access setting**
94+
95+
Go to Security -> Private Access Settings.
96+
97+
* If you've set up private link with your Databricks instance before, you should already have a private access setting configured.
98+
In that case, please ensure that the endpoint allows access to the VPC endpoint created in step (2).
99+
100+
* If this is the first time you are setting PrivateLink for your databricks workspace:
101+
* Click on "Add private access config".
102+
* Please fill in the following details:
103+
* A name for your setting: e.g. "Privatelink settings"
104+
* Your AWS region
105+
* Whether or not to allow public access - only set this as False if all your systems and users access your Databricks workspace through privatelink.
106+
* Private access level - either leave as "Account", or allow-list specific VPCs including the Elementary VPC created in the previous step.
107+
108+
<img
109+
src="https://res.cloudinary.com/dgpojk42n/image/upload/v1763579848/databricks_privatelink_03_add_private_access_setting_rgoisu.png"
110+
alt="Configure Private Access"
111+
/>
112+
113+
4. **Add the private access setting to your Databricks workspace**
114+
115+
__Note__: If you have already set up Privatelink with Databricks in the past, you can skip this step.
116+
117+
Under the Databricks account management portal, go to Workspaces, click on your workspace and then on "Update Workspace".
118+
Then go to "Advanced Configurations", and under "Private Link", please attach the setting created in the previous step.
119+
120+
5. **Add a Databricks environment in Elementary**
121+
122+
After all the previous steps are completed, please reach out to the Elementary team to verify that your Databricks cluster is accessible via PrivateLink.
123+
124+
Once verified, please add a Databricks environment to Elementary by following [this guide](/cloud/integrations/dwh/databricks).
125+
126+
Under the *Add the Elementary IP to allowlist* section, please add the following private subnets *instead* of the IP mentioned there:
127+
* 10.0.1.x
128+
* 10.0.2.x
129+
* 10.0.3.x
130+
131+
61132
### Github Enterprise Server
62133

63134
Coming soon!

docs/snippets/cloud/integrations/athena.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import CreateUserOperation from '/snippets/cloud/integrations/create-user-operation.mdx';
22

3-
You will connect Elementary Cloud to Athena for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)).
3+
This guide contains the necessary steps to connect an Athena environment to your Elementary account.
44

55
<CreateUserOperation />
66

docs/snippets/cloud/integrations/bigquery.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ import CloudServiceAccount from '/snippets/dwh/bigquery/cloud_service_account.md
22
import PermissionsAndSecurity from '/snippets/cloud/integrations/permissions-and-security.mdx';
33
import IpAllowlist from '/snippets/cloud/integrations/ip-allowlist.mdx';
44

5-
You will connect Elementary Cloud to Bigquery for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)).
5+
This guide contains the necessary steps to connect a BigQuery environment to your Elementary account.
66

77
<CloudServiceAccount />
88

docs/snippets/cloud/integrations/clickhouse.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
import CreateUserOperation from '/snippets/cloud/integrations/create-user-operation.mdx';
22
import IpAllowlist from '/snippets/cloud/integrations/ip-allowlist.mdx';
33

4+
This guide contains the necessary steps to connect an Clickhouse environment to your Elementary account.
5+
46
You will connect Elementary Cloud to Clickhouse for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)).
57

68
<CreateUserOperation />

docs/snippets/cloud/integrations/databricks.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ import CreateServicePrincipal from '/snippets/dwh/databricks/create_service_prin
22
import PermissionsAndSecurity from '/snippets/dwh/databricks/databricks_permissions_and_security.mdx';
33
import IpAllowlist from '/snippets/cloud/integrations/ip-allowlist.mdx';
44

5-
You will connect Elementary Cloud to Databricks for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)).
5+
This guide contains the necessary steps to connect a Databricks environment to your Elementary account.
66

77
<CreateServicePrincipal />
88

docs/snippets/cloud/integrations/dremio.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
This guide contains the necessary steps to connect a Dremio environment to your Elementary account.
2+
13
**Note:** We currently support **Dremio Cloud only**. If you are using Dremio Software, please contact us for assistance.
24

35
### Create a user for Elementary cloud

docs/snippets/cloud/integrations/postgres.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import CreateUserOperation from '/snippets/cloud/integrations/create-user-operation.mdx';
22
import IpAllowlist from '/snippets/cloud/integrations/ip-allowlist.mdx';
33

4-
You will connect Elementary Cloud to Postgres for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)).
4+
This guide contains the necessary steps to connect a Postgres environment to your Elementary account.
55

66
<CreateUserOperation />
77

docs/snippets/cloud/integrations/redshift.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import CreateUserOperation from '/snippets/cloud/integrations/create-user-operation.mdx';
22
import IpAllowlist from '/snippets/cloud/integrations/ip-allowlist.mdx';
33

4-
You will connect Elementary Cloud to Redshift for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)).
4+
This guide contains the necessary steps to connect a Redshift environment to your Elementary account.
55

66
<CreateUserOperation />
77

docs/snippets/cloud/integrations/snowflake.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import CreateUserOperationSnowflake from '/snippets/cloud/integrations/create-user-operation-snowflake.mdx';
22

3-
You will connect Elementary Cloud to Snowflake for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)).
3+
This guide contains the necessary steps to connect a Snowflake environment to your Elementary account.
44

55
<CreateUserOperationSnowflake />
66

0 commit comments

Comments
 (0)