docs: Add Snowflake source connector documentation and align behavior details #986
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,209 @@ | ||||||||
| --- | ||||||||
| title: "Ingest data from Snowflake" | ||||||||
| sidebarTitle: Snowflake | ||||||||
| description: "Load data from Snowflake tables into RisingWave using the ADBC connector." | ||||||||
| --- | ||||||||
|
|
||||||||
| This guide describes how to ingest batch data from Snowflake tables into RisingWave using the ADBC (Arrow Database Connectivity) connector. This enables you to create refreshable tables that periodically pull data from Snowflake. | ||||||||
|
|
||||||||
| Snowflake is a cloud-based data warehousing platform that allows for scalable and efficient data storage and analysis. For more information about Snowflake, see [Snowflake official website](https://www.snowflake.com/en/). | ||||||||
|
|
||||||||
| ## Prerequisites | ||||||||
|
|
||||||||
| * A Snowflake account with access to the database and tables you want to ingest. | ||||||||
| * The Snowflake account identifier (e.g., `myaccount.us-east-1`). | ||||||||
| * Valid authentication credentials (username/password, OAuth token, JWT private key, etc.). | ||||||||
| * Network access from RisingWave to your Snowflake instance. | ||||||||
|
|
||||||||
|
Comment on lines
+11
to
+17
There was a problem hiding this comment. Similar to other new features in the documentation (e.g., MySQL table source has "Added in v2.2.0"), this Snowflake source connector should include a version note indicating when it was added to RisingWave. This helps users understand feature availability across different RisingWave versions. The note should be added after the introductory description, following the pattern seen in other connector documentation. |
||||||||
| ## Connecting to Snowflake | ||||||||
|
|
||||||||
| RisingWave supports loading data from Snowflake tables using the `adbc_snowflake` connector. This creates a refreshable table that periodically fetches the latest data from Snowflake. | ||||||||
|
|
||||||||
| ### Syntax | ||||||||
|
|
||||||||
| ```sql | ||||||||
| CREATE TABLE table_name ( | ||||||||
| primary key (order_id) -- Replace with your actual primary key column(s) | ||||||||
| ) WITH ( | ||||||||
| connector = 'adbc_snowflake', | ||||||||
| refresh_mode = 'FULL_RELOAD', | ||||||||
| refresh_interval_sec = 'interval_in_seconds', | ||||||||
| adbc_snowflake.account = 'snowflake_account', | ||||||||
| adbc_snowflake.username = 'username', | ||||||||
| adbc_snowflake.password = 'password', | ||||||||
| adbc_snowflake.database = 'database_name', | ||||||||
| adbc_snowflake.schema = 'schema_name', | ||||||||
| adbc_snowflake.warehouse = 'warehouse_name', | ||||||||
| adbc_snowflake.table = 'table_name' | ||||||||
|
||||||||
| adbc_snowflake.table = 'table_name' | |
| adbc_snowflake.table = 'source_table_name' |
Outdated
There was a problem hiding this comment.
The parameter description for refresh_interval_sec states it's required ("All parameters are required unless specified otherwise"), but this conflicts with the behavior described in the Limitations section and similar connectors like Iceberg, where omitting this parameter means manual refresh only. The description should clarify that this parameter is optional, and if omitted, the table will only refresh when manually triggered via REFRESH TABLE.
| | refresh_interval_sec | The refresh interval in seconds. Determines how frequently data is fetched from Snowflake. | | |
| | refresh_interval_sec | **Optional.** The refresh interval in seconds. Determines how frequently data is fetched from Snowflake. If omitted, the table is not refreshed automatically and only updates when you run `REFRESH TABLE`. | |
Outdated
There was a problem hiding this comment.
The parameter description states that adbc_snowflake.password is "Required if using the default auth_snowflake authentication type", but the default authentication type is described as auth_snowflake on line 63. There's an inconsistency in naming - the description uses "auth_snowflake" but should verify this matches the actual implementation. Based on the parameter name pattern and auth type examples, this should likely be just the default (username/password) rather than specifically named "auth_snowflake".
| | adbc_snowflake.password | **Optional**. The password for username/password authentication. Required if using the default `auth_snowflake` authentication type. | | |
| | adbc_snowflake.password | **Optional**. The password for username/password authentication. Required when using username/password authentication (the default auth type). | |
There was a problem hiding this comment.
The OAuth authentication example sets adbc_snowflake.auth_type = 'auth_oauth' but doesn't include any OAuth-specific parameters such as an OAuth token or token endpoint. OAuth authentication typically requires additional parameters beyond just setting the auth type. The example should either include the required OAuth parameters or add a note explaining what additional configuration is needed for OAuth to work properly.
| Note: This example only shows how to set the authentication type to OAuth. You must also configure the required OAuth credentials and related settings for the ADBC Snowflake connector (for example, via its configuration or environment) for authentication to work correctly. |
There was a problem hiding this comment.
Could you clarify what needs to be changed with the doc ref on line 130? The current reference [Overview of data types](/sql/data-types/overview) follows the same pattern used in other source/sink documentation (e.g., Snowflake sink line 57, SQL Server CDC line 220). Should I use a different link format or reference a different page?
Outdated
There was a problem hiding this comment.
The "What's next?" section links to "Data formats and encoding options", but the Snowflake ADBC connector doesn't use FORMAT/ENCODE options - it relies on automatic schema inference from the Snowflake table. This link may confuse users as it's not relevant to the Snowflake source connector. Consider replacing this with a more relevant link, such as documentation about refreshable tables (REFRESH TABLE command) or other batch ingestion patterns.
| * [Data formats and encoding options](/ingestion/formats-and-encoding-options) - Understand supported data formats |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The support matrix shows "❌" for One-Time Batch ingestion for Snowflake, but the connector supports manual refresh via
REFRESH TABLEcommand (similar to Iceberg). According to the documentation, ifrefresh_interval_secis omitted, the table will only refresh when manually triggered. This is effectively one-time batch ingestion. Consider marking this as "✅" to be consistent with Apache Iceberg (line 288) which also supports FULL_RELOAD mode with manual refresh.