diff --git a/docs/cloud/integrations/security-and-connectivity/aws-privatelink-integration.mdx b/docs/cloud/integrations/security-and-connectivity/aws-privatelink-integration.mdx index 6dae4bab2..6dac348b9 100644 --- a/docs/cloud/integrations/security-and-connectivity/aws-privatelink-integration.mdx +++ b/docs/cloud/integrations/security-and-connectivity/aws-privatelink-integration.mdx @@ -46,7 +46,7 @@ In order to set up a PrivateLink connection with Snowflake, please follow the st 3. **Provide Elementary with the configuration obtained in the previous step.** 1. Elementary will then setup the required infrastructure to connect to Snowflake via PrivateLink. 4. **Add a Snowflake environment in Elementary** - 1. Follow the instructions [here](https://docs.elementary-data.com/cloud/onboarding/connect-data-warehouse) to set up a Snowflake environment in Elementary. + 1. Follow the instructions [here](/cloud/integrations/dwh/snowflake) to set up a Snowflake environment in Elementary. 1. When supplying the account, use `.privatelink` , where the account identifier is the result of the following query: ```sql @@ -58,6 +58,77 @@ In order to set up a PrivateLink connection with Snowflake, please follow the st * 10.0.2.x * 10.0.3.x +### Databricks + +Databricks has support for connecting to AWS-hosted Databricks workspaces via PrivateLink. This setup is entirely managed by Databricks, so Elementary connects with an endpoint service hosted on Databrick's AWS account for this purpose. + +**Note**: +1. You must be a Databricks account admin to perform this setup. +2. Your databricks workspace must be deployed on a customer-managed VPC. PrivateLink is not supported with Databricks-managed VPCs. + +In order to set up a PrivateLink connection with Databricks, please follow the steps below: + +1. **Please provide Elementary with the following details:** + * Your databricks workspace URL. + * Your AWS account ID. + * Your AWS region. + + Elementary will then provide you with a VPC Endpoint ID that will be used in the next step. + +2. **Register your VPC endpoint** + + In the account management portal (not your workspace), go to Security -> Networking -> VPC Endpoints, +then click on the "Register VPC Endpoint" button. + + You should fill in: + 1. A name for the VPC endpoint - e.g. "Elementary". + 2. Your AWS region. + 3. The VPC Endpoint ID provided to you by Elementary. + + Register VPC Endpoint + +3. **Configure a private access setting** + + Go to Security -> Private Access Settings. + + * If you've set up private link with your Databricks instance before, you should already have a private access setting configured. +In that case, please ensure that the endpoint allows access to the VPC endpoint created in step (2). + + * If this is the first time you are setting PrivateLink for your databricks workspace: + * Click on "Add private access config". + * Please fill in the following details: + * A name for your setting: e.g. "Privatelink settings" + * Your AWS region + * Whether or not to allow public access - only set this as False if all your systems and users access your Databricks workspace through privatelink. + * Private access level - either leave as "Account", or allow-list specific VPCs including the Elementary VPC created in the previous step. + + Configure Private Access + +4. **Add the private access setting to your Databricks workspace** + + __Note__: If you have already set up Privatelink with Databricks in the past, you can skip this step. + + Under the Databricks account management portal, go to Workspaces, click on your workspace and then on "Update Workspace". +Then go to "Advanced Configurations", and under "Private Link", please attach the setting created in the previous step. + +5. **Add a Databricks environment in Elementary** + + After all the previous steps are completed, please reach out to the Elementary team to verify that your Databricks cluster is accessible via PrivateLink. + + Once verified, please add a Databricks environment to Elementary by following [this guide](/cloud/integrations/dwh/databricks). + + Under the *Add the Elementary IP to allowlist* section, please add the following private subnets *instead* of the IP mentioned there: + * 10.0.1.x + * 10.0.2.x + * 10.0.3.x + + ### Github Enterprise Server Coming soon! diff --git a/docs/snippets/cloud/integrations/athena.mdx b/docs/snippets/cloud/integrations/athena.mdx index 8bbf98986..c6413242b 100644 --- a/docs/snippets/cloud/integrations/athena.mdx +++ b/docs/snippets/cloud/integrations/athena.mdx @@ -1,6 +1,6 @@ import CreateUserOperation from '/snippets/cloud/integrations/create-user-operation.mdx'; -You will connect Elementary Cloud to Athena for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)). +This guide contains the necessary steps to connect an Athena environment to your Elementary account. diff --git a/docs/snippets/cloud/integrations/bigquery.mdx b/docs/snippets/cloud/integrations/bigquery.mdx index 6e56069c5..2303338bc 100644 --- a/docs/snippets/cloud/integrations/bigquery.mdx +++ b/docs/snippets/cloud/integrations/bigquery.mdx @@ -2,7 +2,7 @@ import CloudServiceAccount from '/snippets/dwh/bigquery/cloud_service_account.md import PermissionsAndSecurity from '/snippets/cloud/integrations/permissions-and-security.mdx'; import IpAllowlist from '/snippets/cloud/integrations/ip-allowlist.mdx'; -You will connect Elementary Cloud to Bigquery for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)). +This guide contains the necessary steps to connect a BigQuery environment to your Elementary account. diff --git a/docs/snippets/cloud/integrations/clickhouse.mdx b/docs/snippets/cloud/integrations/clickhouse.mdx index 21013479c..5905db008 100644 --- a/docs/snippets/cloud/integrations/clickhouse.mdx +++ b/docs/snippets/cloud/integrations/clickhouse.mdx @@ -1,6 +1,8 @@ import CreateUserOperation from '/snippets/cloud/integrations/create-user-operation.mdx'; import IpAllowlist from '/snippets/cloud/integrations/ip-allowlist.mdx'; +This guide contains the necessary steps to connect an Clickhouse environment to your Elementary account. + You will connect Elementary Cloud to Clickhouse for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)). diff --git a/docs/snippets/cloud/integrations/databricks.mdx b/docs/snippets/cloud/integrations/databricks.mdx index e3b513c83..5252f24b1 100644 --- a/docs/snippets/cloud/integrations/databricks.mdx +++ b/docs/snippets/cloud/integrations/databricks.mdx @@ -2,7 +2,7 @@ import CreateServicePrincipal from '/snippets/dwh/databricks/create_service_prin import PermissionsAndSecurity from '/snippets/dwh/databricks/databricks_permissions_and_security.mdx'; import IpAllowlist from '/snippets/cloud/integrations/ip-allowlist.mdx'; -You will connect Elementary Cloud to Databricks for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)). +This guide contains the necessary steps to connect a Databricks environment to your Elementary account. diff --git a/docs/snippets/cloud/integrations/dremio.mdx b/docs/snippets/cloud/integrations/dremio.mdx index 968c67500..b9ce4ec12 100644 --- a/docs/snippets/cloud/integrations/dremio.mdx +++ b/docs/snippets/cloud/integrations/dremio.mdx @@ -1,3 +1,5 @@ +This guide contains the necessary steps to connect a Dremio environment to your Elementary account. + **Note:** We currently support **Dremio Cloud only**. If you are using Dremio Software, please contact us for assistance. ### Create a user for Elementary cloud diff --git a/docs/snippets/cloud/integrations/postgres.mdx b/docs/snippets/cloud/integrations/postgres.mdx index 741ad0b67..3d355cb01 100644 --- a/docs/snippets/cloud/integrations/postgres.mdx +++ b/docs/snippets/cloud/integrations/postgres.mdx @@ -1,7 +1,7 @@ import CreateUserOperation from '/snippets/cloud/integrations/create-user-operation.mdx'; import IpAllowlist from '/snippets/cloud/integrations/ip-allowlist.mdx'; -You will connect Elementary Cloud to Postgres for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)). +This guide contains the necessary steps to connect a Postgres environment to your Elementary account. diff --git a/docs/snippets/cloud/integrations/redshift.mdx b/docs/snippets/cloud/integrations/redshift.mdx index e220cbc7a..f9570fdd1 100644 --- a/docs/snippets/cloud/integrations/redshift.mdx +++ b/docs/snippets/cloud/integrations/redshift.mdx @@ -1,7 +1,7 @@ import CreateUserOperation from '/snippets/cloud/integrations/create-user-operation.mdx'; import IpAllowlist from '/snippets/cloud/integrations/ip-allowlist.mdx'; -You will connect Elementary Cloud to Redshift for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)). +This guide contains the necessary steps to connect a Redshift environment to your Elementary account. diff --git a/docs/snippets/cloud/integrations/snowflake.mdx b/docs/snippets/cloud/integrations/snowflake.mdx index 9248a2ef3..e1cdb930a 100644 --- a/docs/snippets/cloud/integrations/snowflake.mdx +++ b/docs/snippets/cloud/integrations/snowflake.mdx @@ -1,6 +1,6 @@ import CreateUserOperationSnowflake from '/snippets/cloud/integrations/create-user-operation-snowflake.mdx'; -You will connect Elementary Cloud to Snowflake for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)). +This guide contains the necessary steps to connect a Snowflake environment to your Elementary account.