Skip to content

Commit 7a5439e

Browse files
kly-segmentkly-segment
authored
Update snowflake-setup.md
1 parent 77ac16a commit 7a5439e

File tree

1 file changed

+64
-73
lines changed

1 file changed

+64
-73
lines changed

‎src/unify/data-graph/setup-guides/snowflake-setup.md

Lines changed: 64 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -6,81 +6,63 @@ hidden: true
66
redirect_from:
77
- '/unify/linked-profiles/setup-guides/snowflake-setup'
88
---
9+
> warning ""
10+
> Data Graph, Reverse ETL, Profiles Sync require different warehouse permissions.
911
10-
> info "Linked Audiences is in public beta"
11-
> Linked Audiences (with Data Graph, Linked Events) is in public beta, and Segment is actively working on this feature. Some functionality may change before it becomes generally available.
12-
13-
On this page, you'll learn how to connect your Snowflake data warehouse to Segment.
14-
15-
Log in to Snowflake with admin privileges to provide Segment Data Graph with the necessary permissions below.
16-
17-
<!-- remove this and go for it! -->
12+
On this page, you'll learn how to connect your Snowflake data warehouse to Segment for the [Data Graph](/docs/unify/data-graph/data-graph/).
1813

19-
20-
## Required connection settings within Segment
14+
## Snowflake credentials
2115

22-
Segment requires the following settings to connect to your Snowflake warehouse.
16+
Log in to Snowflake with admin privileges to provide the Data Graph with the necessary permissions below. Segment recommends setting up a new Snowflake user and only giving this user permission to access the required databases and schemas.
2317

24-
<img src="/docs/unify/images/snowflake-setup.png" alt="Connect Snowflake to Data Graph" width="5888"/>
18+
## Step 1: Create Segment user and internal database for the Data Graph
2519

26-
- **Account ID**: The Snowflake account ID that uniquely identifies your organization account.
27-
- **Database**: The only database that Segment requires write access to in order to create tables for internal bookkeeping. This database is referred to as `segment_connection_db` in the script below.
28-
- **Warehouse**: The [warehouse](https://docs.snowflake.com/en/user-guide/warehouses){:target="_blank”} in your Snowflake account that you want to use for Segment to run the SQL queries. This warehouse is referred to as `segment_connection_warehouse` in the script below.
29-
- **Username**: The Snowflake user that Segment uses to run SQL in your warehouse. This user is referred to as `segment_connection_username` in the script below.
30-
- **Authentication**: There are 2 supported authentication methods:
31-
1. **Key Pair**: This is the recommended method of authentication. You would need to first create the user and assign it a key pair following the instructions in the [Snowflake docs](https://docs.snowflake.com/en/user-guide/key-pair-auth). Then, follow the Segment docs above to set up Snowflake permissions and set the `segment_connections_username` variable in the SQL script to the user you just created.
32-
2. **Password**: The password of the user above. This password is referred to as `segment_connection_password` in the script below.
20+
The first step is to create a new Segment role and grant it the appropriate permissions. Run the SQL code block below in your SQL worksheet in Snowflake to execute the following tasks:
3321

34-
## Set up Snowflake credentials
35-
36-
Segment recommends setting up a new Snowflake user and only giving this user permissions to access the required databases and schemas.
37-
38-
### Step 1: Create Segment user and internal database
39-
40-
The first step is to create a new Segment role and grant it the appropriate permissions. Run the SQL code block below in your SQL worksheet in Snowflake. It executes the following commands:
41-
42-
- Create a new role and user for Segment Data Graph. This new role will have access to only the datasets you want to access from the Segment Data Graph.
22+
- Create a new role and user for the Segment Data Graph. This new role will only have access to the datasets you provide access to for the Data Graph.
4323
- Grant the Segment user access to the warehouse of your choice. If you'd like to create a new warehouse, uncomment the SQL below.
44-
- Create a new database for Segment Data Graph. **Segment only requires write access to this one database to create a schema for internal bookkeeping, and to store checkpoint tables for the queries that are executed**. Segment recommends creating an empty database for this purpose using the script below. This is also the database you'll be required to specify for the "Database Name" when connecting Snowflake with the Segment app.
24+
- Create a new database for the Data Graph. **Segment requires write access to this database in order to create a schema for internal bookkeeping, and to store checkpoint tables for the queries that are executed. Hence, Segment recommends creating an empty database for this purpose using the script below.** This is also the database you'll be required to specify for the "Database Name" when connecting Snowflake with the Segment app.
4525

4626
> info ""
4727
> The variables specified at the top of the code block with the `SET` command are placeholders and should be updated.
4828
49-
```
29+
```SQL
5030
-- ********** SET UP THE FOLLOWING WAREHOUSE PERMISSIONS **********
51-
-- Edit the following variables
52-
SET segment_connection_username='SEGMENT_LINKED_USER';
53-
SET segment_connection_password='my-safe-password';
54-
SET segment_connection_warehouse='SEGMENT_LINKED_WH';
55-
SET segment_connection_role='SEGMENT_LINKED_ROLE';
5631

57-
-- The DB used for Segment's internal bookkeeping. Note: Use this DB in the connection settings on the Segment app. This is the only DB that Segment requires write access to.
32+
-- Update the following variables
33+
SET segment_connection_username = 'SEGMENT_LINKED_USER';
34+
SET segment_connection_password = 'my-safe-password';
35+
SET segment_connection_warehouse = 'SEGMENT_LINKED_WH';
36+
SET segment_connection_role = 'SEGMENT_LINKED_ROLE';
37+
38+
-- The DB used for Segment's internal bookkeeping.
39+
-- Note: Use this DB in the connection settings on the Segment app. This is the only DB that Segment requires write access to.
5840
SET segment_connection_db = 'SEGMENT_LINKED_PROFILES_DB';
5941

6042
-- ********** [OPTIONAL] UNCOMMENT THE CODE BELOW IF YOU NEED TO CREATE A NEW WAREHOUSE **********
43+
6144
-- CREATE WAREHOUSE IF NOT EXISTS identifier($segment_connection_warehouse)
6245
-- WITH WAREHOUSE_SIZE = 'XSMALL'
6346
-- WAREHOUSE_TYPE = 'STANDARD'
6447
-- AUTO_SUSPEND = 600 -- 5 minutes
6548
-- AUTO_RESUME = TRUE;
6649

67-
6850
-- ********** RUN THE COMMANDS BELOW TO FINISH SETTING UP THE WAREHOUSE PERMISSIONS **********
6951

7052
-- Use admin role for setting grants
7153
USE ROLE ACCOUNTADMIN;
7254

73-
-- Create a role for Segment Data Graph
55+
-- Create a role for the Data Graph
7456
CREATE ROLE IF NOT EXISTS identifier($segment_connection_role)
7557
COMMENT = 'Used for Segment Data Graph';
7658

77-
-- Create a user for Segment Data Graph
59+
-- Create a user for the Data Graph
7860
CREATE USER IF NOT EXISTS identifier($segment_connection_username)
7961
MUST_CHANGE_PASSWORD = FALSE
8062
DEFAULT_ROLE = $segment_connection_role
81-
PASSWORD=$segment_connection_password
82-
COMMENT='Segment Data Graph User'
83-
TIMEZONE='UTC';
63+
PASSWORD = $segment_connection_password
64+
COMMENT = 'Segment Data Graph User'
65+
TIMEZONE = 'UTC';
8466

8567
-- Grant permission to the role to use the warehouse
8668
GRANT USAGE ON WAREHOUSE identifier($segment_connection_warehouse) TO ROLE identifier($segment_connection_role);
@@ -96,18 +78,17 @@ GRANT CREATE SCHEMA ON DATABASE identifier($segment_connection_db) TO ROLE iden
9678

9779
```
9880

99-
### Step 2: Grant read-only access to other databases
81+
## Step 2: Grant read-only access to other databases for the Data Graph
10082

101-
Next, give the Segment role **read-only** access to all the other databases you want to use for Data Graph including the **Profiles Sync database**
83+
Next, give the Segment role **read-only** access to all the other databases you want to use for Data Graph including the Profiles Sync database. Repeat the SQL query below for **each** database you want to use for the Data Graph.
10284

103-
Run the SQL query below for **each** database you want to use for Data Graph. **You may have to re-run this multiple times for each database you want to give access to**.
104-
105-
```
85+
``` SQL
10686

107-
SET segment_connection_role='SEGMENT_LINKED_ROLE';
87+
SET segment_connection_role = 'SEGMENT_LINKED_ROLE';
10888

109-
-- Change this for each DB you want to access and re-run the SQL below.
110-
SET linked_read_only_database='MARKETING_DB';
89+
-- ********** REPEAT THE SQL QUERY BELOW FOR EACH DATABASE YOU WANT TO USE FOR THE DATA GRAPH **********
90+
-- Change this for each DB you want to grant the Data Graph read-only access to
91+
SET linked_read_only_database = 'MARKETING_DB';
11192

11293
GRANT USAGE ON DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
11394
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
@@ -122,16 +103,15 @@ GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN DATABASE identifier($linked_read_on
122103

123104
```
124105

125-
### (Optional) Step 3: Restrict Snowflake schema access
106+
## (Optional) Step 3: Restrict Snowflake schema access
126107

127-
If you want to restrict access to specific [Snowflake schemas and tables](https://docs.snowflake.com/en/user-guide/security-access-control-privileges#table-privileges){:target="_blank”}, run the following commands:
108+
If you want to restrict access to specific [Snowflake schemas and tables](https://docs.snowflake.com/en/user-guide/security-access-control-privileges#table-privileges), then run the following commands:
128109

129-
```
110+
```SQL
130111
-- [Optional] Further restrict access to only specific schemas and tables
131-
SET db='MY_DB';
132-
SET schema='MY_DB.MY_SCHEMA_NAME';
133-
SET segment_connection_role='SEGMENT_LINKED_ROLE';
134-
112+
SET db = 'MY_DB';
113+
SET schema = 'MY_DB.MY_SCHEMA_NAME';
114+
SET segment_connection_role = 'SEGMENT_LINKED_ROLE';
135115

136116
-- View specific schemas in database
137117
GRANT USAGE ON DATABASE identifier($db) TO ROLE identifier($segment_connection_role);
@@ -145,40 +125,32 @@ GRANT SELECT ON FUTURE EXTERNAL TABLES IN SCHEMA identifier($linked_read_only_da
145125
GRANT SELECT ON ALL MATERIALIZED VIEWS IN SCHEMA identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
146126
GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN SCHEMA identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
147127

148-
149128
```
150129

151-
### (If applicable) Step 4: Update user acccess for Segment Reverse ETL schema
130+
## (If applicable) Step 4: Update user acccess for Segment Reverse ETL schema
152131

153132
> warning ""
154-
> This is only applicable if you choose to use an existing database as the Segment connection database that has also been used for Segment Reverse ETL.
155-
156-
Run the following SQL if you run into an error on the Segment app indicating that the user doesn't have sufficient privileges on an existing `_segment_reverse_etl` schema.
157-
158-
If Segment Reverse ETL has ever run in the database you are configuring as the Segment connection database, a Segment-managed schema is already created and you need to provide the new Segment user access to the existing schema.
133+
> This is only applicable if you choose to use an existing database as the Segment connection database that has also been used for [Segment Reverse ETL](/docs/connections/reverse-etl/).
159134
160-
Add the Snowflake table permissions by running the following commands:
135+
If Segment Reverse ETL has ever run in the database you are configuring as the Segment connection database, a Segment-managed schema is already created and you need to provide the new Segment user access to the existing schema. Run the following SQL if you run into an error on the Segment app indicating that the user doesn't have sufficient privileges on an existing `_segment_reverse_etl` schema.
161136

162-
```
137+
``` SQL
163138
-- If you want to use an existing database that already has Segment Reverse ETL schemas, you’ll need to run some additional steps below to grant the role access to the existing schemas.
164139

165140
SET retl_schema = concat($segment_connection_db,'.__segment_reverse_etl');
166-
167141
GRANT USAGE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
168-
169142
GRANT CREATE TABLE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
170-
171143
GRANT SELECT,INSERT,UPDATE,DELETE ON ALL TABLES IN SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
172144

173145
```
174146

175-
### Step 5: Confirm permissions
147+
## Step 5: Confirm permissions
176148

177149
To verify you have set up the right permissions for a specific table, log in with the username and password you created for `SEGMENT_CONNECTION_USERNAME` and run the following command to verify the role you created has the correct permissions. If this command succeeds, you should be able to view the respective table.
178150

179-
```
180-
set segment_connection_role='SEGMENT_LINKED_ROLE';
181-
set linked_read_only_database='YOUR_DB';
151+
``` SQL
152+
set segment_connection_role = 'SEGMENT_LINKED_ROLE';
153+
set linked_read_only_database = 'YOUR_DB';
182154
set table_name = 'YOUR_DB.SCHEMA.TABLE';
183155

184156
USE ROLE identifier($segment_connection_role);
@@ -187,3 +159,22 @@ SHOW SCHEMAS;
187159
SELECT * FROM identifier($table_name) LIMIT 10;
188160

189161
```
162+
## Step 6: Connect your warehouse to the Data Graph
163+
To connect your warehouse to the Data Graph:
164+
165+
1. Navigate to **Unify > Data Graph**. This should be a Unify space with Profiles Sync already set up.
166+
2. Click Connect warehouse.
167+
3. Select Snowflake as your warehouse type.
168+
4. Enter your warehouse credentials. Segment requires the following settings to connect to your Snowflake warehouse.
169+
<img src="/docs/unify/images/snowflake-setup.png" alt="Connect Snowflake to Data Graph" width="5888"/>
170+
171+
- **Account ID**: The Snowflake account ID that uniquely identifies your organization account.
172+
- **Database**: The only database that Segment requires write access to in order to create tables for internal bookkeeping. This database is referred to as `segment_connection_db` in the script below.
173+
- **Warehouse**: The [warehouse](https://docs.snowflake.com/en/user-guide/warehouses){:target="_blank”} in your Snowflake account that you want to use for Segment to run the SQL queries. This warehouse is referred to as `segment_connection_warehouse` in the script below.
174+
- **Username**: The Snowflake user that Segment uses to run SQL in your warehouse. This user is referred to as `segment_connection_username` in the script below.
175+
- **Authentication**: There are 2 supported authentication methods:
176+
- **Key Pair**: This is the recommended method of authentication. You would need to first create the user and assign it a key pair following the instructions in the [Snowflake docs](https://docs.snowflake.com/en/user-guide/key-pair-auth). Then, follow the Segment docs above to set up Snowflake permissions and set the `segment_connections_username` variable in the SQL script to the user you just created.
177+
- **Password**: The password of the user above. This password is referred to as `segment_connection_password` in the script below.
178+
179+
5. Test your connection, then click Save.
180+

0 commit comments

Comments
 (0)