-
Notifications
You must be signed in to change notification settings - Fork 380
Update Snowflake Setup Guide #6874
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 3 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
2112a12
Update snowflake-setup.md page in Data Graphs
uditmehta27 592be5d
Update snowflake-setup.md [netlify-build]
cmastr 5170808
Update snowflake-setup.md [netlify-build]
uditmehta27 1238c8c
[netlify-build] address comments
uditmehta27 3d2c9d6
[netlify-build] fix link for realz
uditmehta27 9d6dbe1
update image [netlify-build]
uditmehta27 399618b
update image
uditmehta27 846b428
nit
uditmehta27 9eed382
update note formatting
pwseg File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -21,7 +21,7 @@ Log in to Snowflake with admin privileges to provide Segment Data Graph with the | |||||
|
||||||
Segment requires the following settings to connect to your Snowflake warehouse. | ||||||
|
||||||
<img src="/docs/unify/images/snowflake-setup.png" alt="Connect Snowflake to the Data Graph" width="5888"/> | ||||||
<img src="/docs/unify/images/snowflake-setup.png" alt="Connect Snowflake to Data Graph" width="5888"/> | ||||||
|
||||||
- **Account ID**: The Snowflake account ID that uniquely identifies your organization account. | ||||||
- **Database Name**: The only database that Segment requires write access to in order to create tables for internal bookkeeping. This database is referred to as `segment_connection_db` in the script below. | ||||||
|
@@ -33,13 +33,15 @@ Segment requires the following settings to connect to your Snowflake warehouse. | |||||
|
||||||
Segment recommends setting up a new Snowflake user and only giving this user permissions to access the required databases and schemas. | ||||||
|
||||||
### Create Segment user and internal database | ||||||
### Step 1: Create Segment user and internal database | ||||||
|
||||||
Use the following steps to set up your Snowflake credentials: | ||||||
The first step is to create a new Segment role and grant it the appropriate permissions. Run the SQL code block below in your SQL worksheet in Snowflake. It executes the following commands: | ||||||
|
||||||
- Create a new role and user for Segment Data Graph. | ||||||
- Create a new role and user for Segment Data Graph. This new role will have access to only the datasets you want to access from the Segment Data Graph. | ||||||
- Grant the Segment user access to the warehouse of your choice. If you'd like to create a new warehouse, uncomment the SQL below. | ||||||
- Create a new database for Segment Data Graph. Segment only requires write access to this one database to create a schema for internal bookkeeping, and to store checkpoint tables for the queries that are executed. Segment recommends creating an empty database for this purpose using the script below. This is also the database you'll be required to specify for the "Database Name" when connecting Snowflake with the Segment app. | ||||||
- Create a new database for Segment Data Graph. **Segment only requires write access to this one database to create a schema for internal bookkeeping, and to store checkpoint tables for the queries that are executed**. Segment recommends creating an empty database for this purpose using the script below. This is also the database you'll be required to specify for the "Database Name" when connecting Snowflake with the Segment app. | ||||||
|
||||||
**Note** - the variables specified at the top of the code block with the `SET` command are placeholders and should be updated. | ||||||
|
||||||
``` | ||||||
-- ********** SET UP THE FOLLOWING WAREHOUSE PERMISSIONS ********** | ||||||
|
@@ -91,11 +93,11 @@ GRANT CREATE SCHEMA ON DATABASE identifier($segment_connection_db) TO ROLE iden | |||||
|
||||||
``` | ||||||
|
||||||
### Grant access to other databases | ||||||
### Step 2: Grant read-only access to other databases | ||||||
|
||||||
Next, give the Segment user **read-only** access to all the other databases you want to use for Data Graph. You must grant access to the Profiles Sync database. | ||||||
Next, give the Segment user **read-only** access to all the other databases you want to use for Data Graph including the **Profiles Sync database** | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
Run the SQL query below for **each** database you want to use for Data Graph: | ||||||
Run the SQL query below for **each** database you want to use for Data Graph. **You may have to re-run this multiple times for each database you want to give access to**. | ||||||
|
||||||
``` | ||||||
|
||||||
|
@@ -117,7 +119,7 @@ GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN DATABASE identifier($linked_read_on | |||||
|
||||||
``` | ||||||
|
||||||
### (Optional) Restrict Snowflake schema access | ||||||
### (Optional) Step 3: Restrict Snowflake schema access | ||||||
|
||||||
If you want to restrict access to specific [Snowflake schemas and tables](https://docs.snowflake.com/en/user-guide/security-access-control-privileges#table-privileges){:target="_blank”}, run the following commands: | ||||||
|
||||||
|
@@ -143,7 +145,7 @@ GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN SCHEMA identifier($linked_read_only | |||||
|
||||||
``` | ||||||
|
||||||
### (If applicable) Update user acccess for Segment Reverse ETL schema | ||||||
### (If applicable) Step 4: Update user acccess for Segment Reverse ETL schema | ||||||
|
||||||
> warning "" | ||||||
> This is only applicable if you choose to use an existing database as the Segment connection database that has also been used for Segment Reverse ETL. | ||||||
|
@@ -157,7 +159,7 @@ Add the Snowflake table permissions by running the following commands: | |||||
``` | ||||||
-- If you want to use an existing database that already has Segment Reverse ETL schemas, you’ll need to run some additional steps below to grant the role access to the existing schemas. | ||||||
|
||||||
SET retl_schema = concat($segment_internal_database,'.__segment_reverse_etl'); | ||||||
SET retl_schema = concat($segment_connection_db,'.__segment_reverse_etl'); | ||||||
|
||||||
GRANT USAGE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role); | ||||||
|
||||||
|
@@ -167,7 +169,7 @@ GRANT SELECT,INSERT,UPDATE,DELETE ON ALL TABLES IN SCHEMA identifier($retl_schem | |||||
|
||||||
``` | ||||||
|
||||||
### Confirm permissions | ||||||
### Step 5: Confirm permissions | ||||||
|
||||||
To verify you have set up the right permissions for a specific table, log in with the username and password you created for `SEGMENT_CONNECTION_USERNAME` and run the following command to verify the role you created has the correct permissions. If this command succeeds, you should be able to view the respective table. | ||||||
|
||||||
|
@@ -179,6 +181,6 @@ set table_name = 'YOUR_DB.SCHEMA.TABLE'; | |||||
USE ROLE identifier($segment_connection_role); | ||||||
USE DATABASE identifier($linked_read_only_database) ; | ||||||
SHOW SCHEMAS; | ||||||
SELECT * FROM identifier($table) LIMIT 10; | ||||||
SELECT * FROM identifier($table_name) LIMIT 10; | ||||||
|
||||||
``` |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add explanations for Authorization as well, its in the default UI now?
What Kathlynn's written before (feel free to rev)
If you are using a key pair, you would need to first create the user and assign it a key pair following the instructions in the Snowflake docs. Then, follow the Segment docs above to set up Snowflake permissions and set the “segment_connections_username” variable in the SQL script to the user you just created.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@hansquaredwang addressed your comment!