Skip to content

Commit 5a86d73

Browse files
Merge pull request #6531 from segmentio/DOC-910
Revamp Snowflake dest docs [DOC-910]
2 parents cf6e848 + d59b4bc commit 5a86d73

File tree

1 file changed

+68
-53
lines changed
  • src/connections/storage/catalog/snowflake

1 file changed

+68
-53
lines changed

src/connections/storage/catalog/snowflake/index.md

Lines changed: 68 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -7,22 +7,37 @@ redirect_from:
77

88
{% include content/warehouse-ip.html %}
99

10-
[Snowflake](https://docs.snowflake.net/manuals/index.html) is a data warehouse built for the cloud. Snowflake delivers performance, simplicity, concurrency and affordability.
10+
[Snowflake](https://docs.snowflake.net/manuals/index.html){:target="_blank"} is a data warehouse, built for the cloud, that delivers performance, simplicity, concurrency and affordability.
1111

12-
## Getting Started
12+
## Getting started
1313

14-
There are six steps to get started using Snowflake with Segment. Make sure that you are running the commands in each step while logged in as an `ACCOUNTADMIN`, or an account that has `MANAGE GRANTS`. While Segment uses predefined user (`SEGMENT_USER`), role (`SEGMENT`), warehouse (`SEGMENT_WAREHOUSE`) and database (`SEGMENT_EVENTS`) names, you can use any names you like.
14+
There are six steps to get started using Snowflake with Segment.
1515

16-
1. Create Virtual Warehouse
17-
2. Create Database
18-
3. Create Role for Segment
19-
4. Create User for Segment
20-
5. Test the User and Credentials
21-
6. Connect Snowflake to Segment
16+
1. [Create a virtual warehouse](#step-1-create-a-virtual-warehouse)
17+
2. [Create a database](#step-2-create-database)
18+
3. [Create a role for Segment](#step-3-create-role-for-segment)
19+
4. [Create a user for Segment](#step-4-create-user-for-segment)
20+
5. [Test the user and credentials](#step-5-test-the-user-and-credentials)
21+
6. [Connect Snowflake to Segment](#step-6-connect-snowflake-to-segment)
2222

23-
### Create Virtual Warehouse
23+
### Prerequisites
2424

25-
The Segment Snowflake destination requires a Snowflake [virtual warehouse](https://docs.snowflake.net/manuals/user-guide/warehouses.html){:target="_blank"} to load data in to. To avoid conflicts with other regular operations in your cluster, Segment recommends that you create a new warehouse just for Segment loads, but this is not mandatory. An X-Small warehouse works for most customers when starting.
25+
To set up the virtual warehouse, database, role, and user in Snowflake for Segment's Snowflake destination, you must have the `ACCOUNTADMIN` role, or, a custom role with the following [Snowflake privileges](https://docs.snowflake.com/en/user-guide/security-access-control-overview#label-access-control-overview-privileges){:target="_blank"}:
26+
27+
- [CREATE WAREHOUSE](https://docs.snowflake.com/en/sql-reference/sql/create-warehouse#access-control-requirements){:target="_blank"}: Used to create a Segment-specific virtual warehouses
28+
- [CREATE DATABASE](https://docs.snowflake.com/en/sql-reference/sql/create-database#access-control-requirements){:target="_blank"}: Used to create a Segment-specific database
29+
- [CREATE ROLE](https://docs.snowflake.com/en/sql-reference/sql/create-role#access-control-requirements){:target="_blank"}: Used to create the role that the Segment user assumes in your Snowflake instance
30+
- [CREATE USER](https://docs.snowflake.com/en/sql-reference/sql/create-user#access-control-requirements){:target="_blank"}: Used to create the Segment user in your Snowflake instance
31+
32+
To set up the Snowflake storage destination in Segment, you must have either a [role in the Segment app](/docs/segment-app/iam/roles/) of _Workspace Owner_ or, for Business Tier users, _Warehouse Destination Admin_.
33+
34+
### Step 1: Create a virtual warehouse
35+
36+
Segment's Snowflake destination requires you to first create a Snowflake [virtual warehouse](https://docs.snowflake.com/en/user-guide/warehouses){:target="_blank"}.
37+
38+
To avoid conflicts with other operations in your cluster, Segment recommends that you create a new warehouse just for Segment loads. An X-Small warehouse is large enough for most Segment customers when they first create their Snowflake destination.
39+
40+
To create a new virtual warehouse, navigate to **Warehouses** > **Create** in Snowflake's Classic Console or execute the following SQL command:
2641

2742
```sql
2843
CREATE WAREHOUSE "SEGMENT_WAREHOUSE"
@@ -32,45 +47,46 @@ CREATE WAREHOUSE "SEGMENT_WAREHOUSE"
3247
AUTO_RESUME = TRUE;
3348
```
3449

35-
Make sure `AUTO_SUSPEND` is set to ~10 minutes in the UI (or 600 if using SQL) and `AUTO_RESUME` is enabled, to avoid extra costs.
50+
> success ""
51+
> Set `AUTO_SUSPEND` to ~10 minutes in the UI (or 600 if using SQL) and enable `AUTO_RESUME` to avoid extra costs, as Snowflake uses [per-second billing](https://docs.snowflake.com/en/user-guide/warehouses-considerations#automating-warehouse-suspension){:target="_blank"}.
52+
53+
### Step 2: Create a database
3654

37-
### Create Database
55+
Segment recommends creating a new database just for Segment information, as the Segment Snowflake destination creates its own schemas and tables and could create name conflicts with your existing data.
3856

39-
The Segment Snowflake destination creates its own schemas and tables, so it's recommended to create a new database for this purpose to avoid name conflicts with existing data.
57+
To create a new database, execute the following SQL command:
4058

4159
```sql
4260
CREATE DATABASE "SEGMENT_EVENTS";
4361
```
4462

45-
### Create Role for Segment
63+
### Step 3: Create a role for Segment
4664

47-
You need to run these commands rather than creating a role with the "Create Role" dialog in the UI.
65+
You need to run these SQL commands rather than creating a role with the "Create Role" dialog in the Classic Console UI.
4866

49-
This role will be attached to Segment's user and it gives just enough permissions for loading data in your database. Segment recommends that you not reuse this role for other operations.
50-
51-
1. Click on to Worksheets;
67+
This role gives Segment just enough permission to load data into your database. Segment recommends that you don't reuse this role for other operations.
68+
1. Click on **Worksheets**
5269
2. Select SEGMENT_EVENTS under database objects
53-
3. Change role to ACCOUNTADMIN
54-
55-
4. Create a new role using the following command:
70+
3. Change the role to `ACCOUNTADMIN`
71+
4. Create a new role by executing the following command:
5672
```sql
5773
CREATE ROLE "SEGMENT";
5874
```
5975

60-
5. Grant access to the virtual warehouse:
76+
5. Grant access to the virtual warehouse by executing the following SQL command:
6177
```sql
6278
GRANT USAGE ON WAREHOUSE "SEGMENT_WAREHOUSE" TO ROLE "SEGMENT";
6379
```
6480

65-
6. Grant access to the database:
81+
6. Grant access to the database by executing the following SQL command:
6682
```sql
6783
GRANT USAGE ON DATABASE "SEGMENT_EVENTS" TO ROLE "SEGMENT";
6884
GRANT CREATE SCHEMA ON DATABASE "SEGMENT_EVENTS" TO ROLE "SEGMENT";
6985
```
7086

71-
### Create User for Segment
87+
### Step 4: Create a user for Segment
7288

73-
Finally, you need to create the user that will be connected to Segment. Be sure to use a strong, unique password.
89+
Create the user that Segment uses to connect to your warehouse. Be sure to use a strong, unique password.
7490

7591
```sql
7692
CREATE USER "SEGMENT_USER"
@@ -80,28 +96,28 @@ CREATE USER "SEGMENT_USER"
8096
GRANT ROLE "SEGMENT" TO USER "SEGMENT_USER";
8197
```
8298

83-
### Test the User and Credentials
99+
### Step 5: Test the user and credentials
84100

85101
Before you continue, test and validate the new user and credentials. When you can run the following commands successfully, you can connect Snowflake to Segment.
86102

87-
Segment uses [snowsql](https://docs.snowflake.net/manuals/user-guide/snowsql.html){:target="_blank"} to run these verification steps.
88-
To install and verify your accounts:
103+
Segment uses [SnowSQL](https://docs.snowflake.com/en/user-guide/snowsql){:target="_blank"} to run these verification steps.
104+
To install SnowSQL and verify your accounts:
89105

90-
1. Download [snowsql](https://docs.snowflake.net/manuals/user-guide/snowsql.html){:target="_blank"}
106+
1. Download [SnowSQL](https://docs.snowflake.com/en/user-guide/snowsql){:target="_blank"}
91107
2. Open the Installer and follow instructions
92-
3. Once the installation is complete, run the following command, replacing "account" and "user" with your Snowflake Account and username:
108+
3. When the installation is complete, run the following command, replacing "account" and "user" with your Snowflake Account ID and username:
93109

94110
```
95111
snowsql -a <account> -u <user>
96112
```
97113

98114
For accounts outside the US, the account ID includes the region. You can find your account name from the browser address string.
99115

100-
If your web address is `https://myaccountname.snowflakecomputing.com/console#/internal/worksheet`, your account name would be `myaccountname`.
116+
For example, if your web address is `https://myaccountname.snowflakecomputing.com/console#/internal/worksheet`, your account name would be `myaccountname`.
101117

102118
You can also find part of your account name by running the following query on your worksheet in Snowflake:
103119

104-
```
120+
```sql
105121
SELECT CURRENT_ACCOUNT();
106122
```
107123
4. Enter password when prompted.
@@ -145,55 +161,54 @@ USE WAREHOUSE "SEGMENT_WAREHOUSE";
145161
USE DATABASE "SEGMENT_EVENTS";
146162
```
147163

148-
### Connect Snowflake to Segment
164+
### Step 6: Connect Snowflake to Segment
149165

150-
After creating a Snowflake warehouse, the next step is to connect Segment.
166+
After configuring your Snowflake resources, connect them to Segment.
151167

152168
1. In the Segment App, select Add Destination.
153169
2. Search for and select "Snowflake".
154170
3. Add your credentials as follows:
155-
- User - The user name (as created above).
156-
- Password - The password for the user.
157-
- Account - The account id of your cluster, not the url (for example, url: `my-business.snowflakecomputing.com`, account-id: `my-business`. **Note:** If you are using Snowflake on AWS, the account id includes the region, for example your url might look like: `my-business.us-east-1.snowflakecomputing.com/` and your accound-id would be: `my-business.us-east-1`)
158-
- Database - The database name (as created above).
159-
- Warehouse - The warehouse name (as created above).
171+
- **User**: The user name that you created in [Step 4: Create a user for Segment](#step-4-create-user-for-segment)
172+
- **Password**: The password that you set in [Step 4: Create a user for Segment](#step-4-create-user-for-segment)
173+
- **Account**: The account id of your cluster, not the url (for example, url: `my-business.snowflakecomputing.com`, account-id: `my-business`. **Note:** If you are using Snowflake on AWS, the account id includes the region. For example, your url might be: `my-business.us-east-1.snowflakecomputing.com/` and your account-id would be: `my-business.us-east-1`)
174+
- **Database**: The database name that you created in [Step 2: Create database](#step-2-create-database)
175+
- **Warehouse**: The name of the warehouse that you created in [Step 1: Create a virtual warehouse](#step-1-create-a-virtual-warehouse)
160176

161177
## Security
162178

163179
### Allowlisting IPs
164180

165-
If you create a network policy with Snowflake, add the following IP addresses to the "Allowed IP Addresses" list: `52.25.130.38/32`, `34.223.203.0/28`
181+
If you create a network policy with Snowflake and are located in the US, add `52.25.130.38/32` and `34.223.203.0/28` to the "Allowed IP Addresses" list.
182+
183+
If you create a network policy with Snowflake and are located in the EU, add `3.251.148.96/29` to your "Allowed IP Addresses" list.
166184

167185
### Multi-Factor Authentication (MFA) & SSO
168186

169187
At this time, the Segment Snowflake destination is not compatible with Snowflake's MFA or SSO settings. If your connected user has MFA or SSO enabled, you will need to disable it for syncs to run correctly.
170188

171189
## Best Practices
172190

173-
### Auto Suspend
174-
175-
Set `AUTO_SUSPEND` to ~10 minutes in the UI (or 600 if using SQL) to avoid credit consumption by the Segment syncing process.
176-
191+
### Auto Suspend and Auto Resume
177192

178-
### Auto Resume
193+
Set `AUTO_SUSPEND` to ~10 minutes in the UI (or 600 if using SQL) to minimize the credit consumption of Segment's syncing process.
179194

180195
If you enable the `AUTO_SUSPEND` feature, Segment recommends that you also enable `AUTO-RESUME`. This will ensure that your Snowflake warehouse automatically resumes when Segment loads data. Otherwise, Segment will not be able to load data unless you [manually resume your Snowflake warehouse](https://docs.snowflake.net/manuals/user-guide/warehouses-considerations.html#automating-warehouse-resumption){:target="_blank"}.
181196

182197
### Unique Warehouse, Database, and Role
183198

184-
Segment recommends creating a unique Warehouse, Database and Role for the Segment Snowflake connection to your Snowflake instance.
199+
Segment recommends creating a unique Warehouse, Database, and Role for the Segment Snowflake connection to your Snowflake instance to avoid conflicts with other operations happening in your cluster.
185200

186201
## Troubleshooting
187202

188203
### I get "Object does not exist" when running "USE DATABASE" or "USE WAREHOUSE", even if the warehouse or the database are created.
189204

190-
Make sure you have created the role and assigned the proper permissions with the account `SYSADMIN` or `ACCOUNTADMIN`. Other non-system accounts don't assign the right permissions.
205+
Make sure you created the role and assigned the proper permissions with the account `SYSADMIN` or `ACCOUNTADMIN`. Other non-system accounts don't assign the right permissions.
191206

192-
### I've consumed all the credits after the initial sync.
207+
### I've consumed all my credits after the initial sync.
193208

194-
If you have used all your credits, you will need to contact Snowflake to purchase more.
209+
If you have used all your credits, you must contact Snowflake to purchase more.
195210

196-
Also make sure `AUTO_SUSPEND` is enabled and set to 5 or 10 minutes in the warehouse used by Segment. This setting will help avoid unintended use of credits by the Segment Snowflake destination.
211+
Also, make sure `AUTO_SUSPEND` is enabled and set to 5 or 10 minutes in the warehouse used by Segment. This setting helps avoid unintended use of credits by the Segment Snowflake destination.
197212

198213
### My syncs are going slower than I expect.
199214

@@ -212,10 +227,10 @@ Most customers have the best luck starting with a X-Small instance.
212227
A `rollback` is issued at the end of each session to make sure there's no "in-flight" processes hanging out that could block other processes later.
213228

214229
### Does Segment use transactions for loading data?
215-
Segment doesn't open transactions explicitly because that would lock resources. However, if autocommit is enabled, each statement functions as it's own transaction, and a silent commit is issued after each.
230+
Segment doesn't open transactions explicitly because that would lock resources. However, if `autocommit` is enabled, each statement functions as its own transaction, and a silent commit is issued after each.
216231

217232
### What privileges do I need to grant?
218-
You shouldn't need to grant any additional privileges. However, you may need to confirm that the USAGE privilege on those schemas is granted to the same role granted to the user connecting to Snowflake through data bricks.
233+
You shouldn't need to grant any additional privileges. However, you may need to confirm that the USAGE privilege on those schemas is granted to the same role granted to the user connecting to Snowflake through Databricks.
219234

220235
Run these statements in Snowflake UI or CLI, and check the output to verify the permissions.
221236

0 commit comments

Comments
 (0)