From 6074b3493d4d4b86d7bfe331c89c15e91462a234 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 13 Feb 2026 19:57:14 +0000 Subject: [PATCH 1/6] Initial plan From 161ce97657497effcd8a1c2e3c7862dea1a966a5 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 13 Feb 2026 20:00:42 +0000 Subject: [PATCH 2/6] Add Snowflake source documentation with cross-links Co-authored-by: kwannoel <47273164+kwannoel@users.noreply.github.com> --- docs.json | 3 +- iceberg/integ-snowflake.mdx | 6 + ingestion/overview.mdx | 2 + ingestion/sources/snowflake.mdx | 208 ++++++++++++++++++++++++ integrations/destinations/snowflake.mdx | 2 + 5 files changed, 220 insertions(+), 1 deletion(-) create mode 100644 ingestion/sources/snowflake.mdx diff --git a/docs.json b/docs.json index b308770b..d9b35c86 100644 --- a/docs.json +++ b/docs.json @@ -184,7 +184,8 @@ "group": "Database tables (non-CDC)", "pages": [ "integrations/sources/postgresql-table", - "integrations/sources/mysql-table" + "integrations/sources/mysql-table", + "ingestion/sources/snowflake" ] }, { diff --git a/iceberg/integ-snowflake.mdx b/iceberg/integ-snowflake.mdx index b8d710f0..4ee48e69 100644 --- a/iceberg/integ-snowflake.mdx +++ b/iceberg/integ-snowflake.mdx @@ -6,6 +6,12 @@ description: "Sink data from RisingWave to an Apache Iceberg table and query it This guide shows how to sink data from RisingWave into an Apache Iceberg table and make it available for querying in Snowflake. This integration allows you to use RisingWave for real-time stream processing and Snowflake for large-scale analytics and data warehousing. + +For direct Snowflake integration, see: +- [Ingest data from Snowflake](/ingestion/sources/snowflake) - Load data from Snowflake tables into RisingWave +- [Sink data to Snowflake](/integrations/destinations/snowflake) - Write data from RisingWave to Snowflake + + **How it works** RisingWave → Iceberg table on S3 → AWS Glue or REST Catalog → Snowflake diff --git a/ingestion/overview.mdx b/ingestion/overview.mdx index 52d9a084..e5f6532f 100644 --- a/ingestion/overview.mdx +++ b/ingestion/overview.mdx @@ -36,6 +36,7 @@ Below is a complete list of source connectors in RisingWave. Click a connector n | [Webhook](/integrations/sources/webhook) | Built-in | | [Events API (HTTP)](/integrations/sources/events-api) | External service | | [Apache Iceberg](/ingestion/sources/iceberg) | | +| [Snowflake](/ingestion/sources/snowflake) | Latest | | [Load generator (datagen)](/ingestion/sources/datagen) | Built-in | For information on supported data formats and encodings, and whether you need to use `CREATE SOURCE` or `CREATE TABLE` with each format, see [Data formats and encoding options](/ingestion/formats-and-encoding-options). @@ -285,6 +286,7 @@ CREATE TABLE test_users ( | **Google Cloud Storage** | ❌ | ✅ | ⚠️ | Batch only; periodic via external tools | | **Azure Blob** | ❌ | ✅ | ⚠️ | Batch only; periodic via external tools | | **Apache Iceberg** | ❌ | ✅ | ⚠️ | Batch only; periodic via external tools | +| **Snowflake** | ❌ | ❌ | ✅ | Periodic refresh with `refresh_interval_sec` | | **Datagen** | ✅ | ❌ | ❌ | Test data generation only | | **Direct INSERT** | ❌ | ✅ | ⚠️ | Manual insertion; periodic via external tools | | **Webhook** | ✅ | ✅ | ⚠️ | Push-based HTTP ingestion; best for SaaS webhooks + request validation/signatures | diff --git a/ingestion/sources/snowflake.mdx b/ingestion/sources/snowflake.mdx new file mode 100644 index 00000000..f39bb751 --- /dev/null +++ b/ingestion/sources/snowflake.mdx @@ -0,0 +1,208 @@ +--- +title: "Ingest data from Snowflake" +sidebarTitle: Snowflake +description: "Load data from Snowflake tables into RisingWave using the ADBC connector." +--- + +This guide describes how to ingest batch data from Snowflake tables into RisingWave using the ADBC (Arrow Database Connectivity) connector. This enables you to create refreshable tables that periodically pull data from Snowflake. + +Snowflake is a cloud-based data warehousing platform that allows for scalable and efficient data storage and analysis. For more information about Snowflake, see [Snowflake official website](https://www.snowflake.com/en/). + +## Prerequisites + +* A Snowflake account with access to the database and tables you want to ingest. +* The Snowflake account identifier (e.g., `myaccount.us-east-1`). +* Valid authentication credentials (username/password, OAuth token, JWT private key, etc.). +* Network access from RisingWave to your Snowflake instance. + +## Connecting to Snowflake + +RisingWave supports loading data from Snowflake tables using the `adbc_snowflake` connector. This creates a refreshable table that periodically fetches the latest data from Snowflake. + +### Syntax + +```sql +CREATE TABLE table_name ( + primary key (column_name) +) WITH ( + connector = 'adbc_snowflake', + refresh_mode = 'FULL_RELOAD', + refresh_interval_sec = 'interval_in_seconds', + adbc_snowflake.account = 'snowflake_account', + adbc_snowflake.username = 'username', + adbc_snowflake.password = 'password', + adbc_snowflake.database = 'database_name', + adbc_snowflake.schema = 'schema_name', + adbc_snowflake.warehouse = 'warehouse_name', + adbc_snowflake.table = 'table_name' +); +``` + + +**Automatic Schema Inference** + +Do not manually specify column definitions in the `CREATE TABLE` statement. The schema is automatically inferred from the Snowflake table. + + +## Parameters + +All parameters are required unless specified otherwise. + +| Parameter | Description | +| :--------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| connector | Must be `adbc_snowflake`. | +| refresh_mode | Must be `FULL_RELOAD`. The entire table is re-read on each refresh. | +| refresh_interval_sec | The refresh interval in seconds. Determines how frequently data is fetched from Snowflake. | +| adbc_snowflake.account | The Snowflake account identifier (e.g., `myaccount.us-east-1` or `myaccount`). | +| adbc_snowflake.username | The Snowflake username for authentication. | +| adbc_snowflake.password | **Optional**. The password for username/password authentication. Required if using the default `auth_snowflake` authentication type. | +| adbc_snowflake.database | The name of the Snowflake database. | +| adbc_snowflake.schema | The Snowflake schema containing the table. | +| adbc_snowflake.warehouse | The Snowflake warehouse to use for queries. | +| adbc_snowflake.table | The name of the Snowflake table to ingest. | +| adbc_snowflake.auth_type | **Optional**. The authentication method. Default is `auth_snowflake` (username/password). Other options: `auth_oauth`, `auth_jwt`, `auth_ext_browser`, `auth_okta`, `auth_mfa`, `auth_pat`, `auth_wif`. | +| adbc_snowflake.jwt_private_key_path | **Optional**. Path to the JWT private key file. Required when using `auth_jwt` authentication. | + +## Authentication methods + +The Snowflake connector supports multiple authentication methods: + +### Username and password (default) + +```sql +CREATE TABLE my_snowflake_table ( + primary key ("order_id") +) WITH ( + connector = 'adbc_snowflake', + refresh_mode = 'FULL_RELOAD', + refresh_interval_sec = '3600', + adbc_snowflake.account = 'myaccount.us-east-1', + adbc_snowflake.username = 'myuser', + adbc_snowflake.password = 'mypassword', + adbc_snowflake.database = 'SALES_DB', + adbc_snowflake.schema = 'PUBLIC', + adbc_snowflake.warehouse = 'COMPUTE_WH', + adbc_snowflake.table = 'ORDERS' +); +``` + +### JWT authentication + +```sql +CREATE TABLE my_snowflake_table ( + primary key ("order_id") +) WITH ( + connector = 'adbc_snowflake', + refresh_mode = 'FULL_RELOAD', + refresh_interval_sec = '7200', + adbc_snowflake.account = 'myaccount', + adbc_snowflake.username = 'myuser', + adbc_snowflake.database = 'SALES_DB', + adbc_snowflake.schema = 'PUBLIC', + adbc_snowflake.warehouse = 'COMPUTE_WH', + adbc_snowflake.table = 'ORDERS', + adbc_snowflake.auth_type = 'auth_jwt', + adbc_snowflake.jwt_private_key_path = '/path/to/key.pem' +); +``` + +### OAuth authentication + +```sql +CREATE TABLE my_snowflake_table ( + primary key ("order_id") +) WITH ( + connector = 'adbc_snowflake', + refresh_mode = 'FULL_RELOAD', + refresh_interval_sec = '3600', + adbc_snowflake.account = 'myaccount.us-east-1', + adbc_snowflake.username = 'myuser', + adbc_snowflake.database = 'SALES_DB', + adbc_snowflake.schema = 'PUBLIC', + adbc_snowflake.warehouse = 'COMPUTE_WH', + adbc_snowflake.table = 'ORDERS', + adbc_snowflake.auth_type = 'auth_oauth' +); +``` + +## Data type mapping + +The following table shows the corresponding data types between Snowflake and RisingWave. For details on native RisingWave data types, see [Overview of data types](/sql/data-types/overview). + +| Snowflake type | RisingWave type | Notes | +| :--------------- | :------------------------------- | :------------------------------------------------- | +| STRING | VARCHAR | | +| NUMBER | DECIMAL or BIGINT | Depends on scale and precision | +| FLOAT | DOUBLE PRECISION | | +| DECIMAL | DECIMAL | | +| CHAR | VARCHAR | | +| TEXT | VARCHAR | | +| DATE | DATE | | +| TIME | Not supported | Will report an error | +| TIMESTAMP_NTZ | TIMESTAMP WITHOUT TIME ZONE | | +| TIMESTAMP_LTZ | TIMESTAMP WITH TIME ZONE | | +| TIMESTAMP_TZ | TIMESTAMP WITH TIME ZONE | | +| BOOLEAN | BOOLEAN | | +| BINARY | BYTEA | | +| VARIANT | VARCHAR | JSON data stored as string | +| OBJECT | VARCHAR | JSON objects stored as string | +| ARRAY | VARCHAR | Arrays stored as string | + +## Complete example + +This example demonstrates how to create a refreshable table that loads data from a Snowflake table every hour. + +### Step 1: Create the refreshable table + +```sql +CREATE TABLE snowflake_orders ( + primary key ("order_id") +) WITH ( + connector = 'adbc_snowflake', + refresh_mode = 'FULL_RELOAD', + refresh_interval_sec = '3600', -- Refresh every hour + + -- Snowflake connection parameters + adbc_snowflake.account = 'myaccount.us-east-1', + adbc_snowflake.username = 'analytics_user', + adbc_snowflake.password = 'secure_password', + adbc_snowflake.database = 'PRODUCTION', + adbc_snowflake.schema = 'SALES', + adbc_snowflake.warehouse = 'ANALYTICS_WH', + adbc_snowflake.table = 'ORDERS' +); +``` + +### Step 2: Query the data + +```sql +SELECT * FROM snowflake_orders LIMIT 10; +``` + +### Step 3: Create materialized views + +You can create materialized views based on the Snowflake data: + +```sql +CREATE MATERIALIZED VIEW daily_sales AS +SELECT + DATE_TRUNC('day', order_date) AS sale_date, + COUNT(*) AS order_count, + SUM(total_amount) AS total_revenue +FROM snowflake_orders +GROUP BY DATE_TRUNC('day', order_date); +``` + +## Limitations and requirements + +* **Refresh mode**: Only `FULL_RELOAD` mode is supported. The entire table is re-read on each refresh interval. +* **Schema inference**: Column definitions are automatically inferred from the Snowflake table. Do not manually specify columns in the `CREATE TABLE` statement. +* **Feature flag**: The Snowflake connector requires the `source-adbc_snowflake` feature to be enabled at compile time. This is enabled by default in official RisingWave builds. +* **Consistent snapshots**: The connector uses Snowflake's time travel feature to ensure all data is read from the same point in time during a refresh. +* **Performance**: For large tables, consider the refresh interval carefully to balance data freshness with query costs in Snowflake. + +## What's next? + +* [Sink data to Snowflake](/integrations/destinations/snowflake) - Learn how to write data from RisingWave back to Snowflake +* [Work with Snowflake and Iceberg](/iceberg/integ-snowflake) - Integrate Snowflake with Apache Iceberg catalogs +* [Data formats and encoding options](/ingestion/formats-and-encoding-options) - Understand supported data formats diff --git a/integrations/destinations/snowflake.mdx b/integrations/destinations/snowflake.mdx index f9e618c1..3b48fca0 100644 --- a/integrations/destinations/snowflake.mdx +++ b/integrations/destinations/snowflake.mdx @@ -6,6 +6,8 @@ description: This guide describes how to sink data from RisingWave to Snowflake Snowflake is a cloud-based data warehousing platform that allows for scalable and efficient data storage and analysis. For more information about Snowflake, see [Snowflake official website](https://www.snowflake.com/en/). +This page describes how to **sink data to** Snowflake. To **ingest data from** Snowflake, see [Ingest data from Snowflake](/ingestion/sources/snowflake). + Sinking from RisingWave to Snowflake utilizes [Snowpipe](https://docs.snowflake.com/en/user-guide/data-load-snowpipe-rest-apis) for data loading. Initially, data is staged in a user-managed S3 bucket in JSON format, and then loaded into the Snowflake table via Snowpipe. For more information, see [Overview of the Snowpipe REST endpoints to load data](https://docs.snowflake.com/user-guide/data-load-snowpipe-rest-overview). From c39d20cdeeec9024b66c31038a065e6a72c101e9 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 13 Feb 2026 20:02:19 +0000 Subject: [PATCH 3/6] Fix documentation clarity issues in Snowflake source Co-authored-by: kwannoel <47273164+kwannoel@users.noreply.github.com> --- ingestion/sources/snowflake.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ingestion/sources/snowflake.mdx b/ingestion/sources/snowflake.mdx index f39bb751..46200c83 100644 --- a/ingestion/sources/snowflake.mdx +++ b/ingestion/sources/snowflake.mdx @@ -41,7 +41,7 @@ CREATE TABLE table_name ( **Automatic Schema Inference** -Do not manually specify column definitions in the `CREATE TABLE` statement. The schema is automatically inferred from the Snowflake table. +Column definitions are automatically inferred from the Snowflake table and should not be manually specified in the `CREATE TABLE` statement. However, you must specify the primary key if your table requires one. ## Parameters @@ -61,7 +61,7 @@ All parameters are required unless specified otherwise. | adbc_snowflake.warehouse | The Snowflake warehouse to use for queries. | | adbc_snowflake.table | The name of the Snowflake table to ingest. | | adbc_snowflake.auth_type | **Optional**. The authentication method. Default is `auth_snowflake` (username/password). Other options: `auth_oauth`, `auth_jwt`, `auth_ext_browser`, `auth_okta`, `auth_mfa`, `auth_pat`, `auth_wif`. | -| adbc_snowflake.jwt_private_key_path | **Optional**. Path to the JWT private key file. Required when using `auth_jwt` authentication. | +| adbc_snowflake.jwt_private_key_path | Required when `adbc_snowflake.auth_type` is `auth_jwt`. Path to the JWT private key file. | ## Authentication methods From 9d1880e0833ac7a1713e877f485661141d76af30 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 13 Feb 2026 20:03:19 +0000 Subject: [PATCH 4/6] Improve Snowflake source documentation clarity and examples Co-authored-by: kwannoel <47273164+kwannoel@users.noreply.github.com> --- ingestion/sources/snowflake.mdx | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/ingestion/sources/snowflake.mdx b/ingestion/sources/snowflake.mdx index 46200c83..7341b4ee 100644 --- a/ingestion/sources/snowflake.mdx +++ b/ingestion/sources/snowflake.mdx @@ -23,7 +23,7 @@ RisingWave supports loading data from Snowflake tables using the `adbc_snowflake ```sql CREATE TABLE table_name ( - primary key (column_name) + primary key (order_id) -- Replace with your actual primary key column(s) ) WITH ( connector = 'adbc_snowflake', refresh_mode = 'FULL_RELOAD', @@ -61,7 +61,7 @@ All parameters are required unless specified otherwise. | adbc_snowflake.warehouse | The Snowflake warehouse to use for queries. | | adbc_snowflake.table | The name of the Snowflake table to ingest. | | adbc_snowflake.auth_type | **Optional**. The authentication method. Default is `auth_snowflake` (username/password). Other options: `auth_oauth`, `auth_jwt`, `auth_ext_browser`, `auth_okta`, `auth_mfa`, `auth_pat`, `auth_wif`. | -| adbc_snowflake.jwt_private_key_path | Required when `adbc_snowflake.auth_type` is `auth_jwt`. Path to the JWT private key file. | +| adbc_snowflake.jwt_private_key_path | Required when `adbc_snowflake.auth_type` is `auth_jwt`. Local file path on the RisingWave server to the JWT private key file (e.g., `/path/to/key.pem`). | ## Authentication methods @@ -184,6 +184,7 @@ SELECT * FROM snowflake_orders LIMIT 10; You can create materialized views based on the Snowflake data: ```sql +-- The columns order_date and total_amount are automatically inferred from the Snowflake table CREATE MATERIALIZED VIEW daily_sales AS SELECT DATE_TRUNC('day', order_date) AS sale_date, From 4ffdd410fdaa3e4057a0ffe5f397d22d3202bddf Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 26 Feb 2026 09:43:19 +0000 Subject: [PATCH 5/6] Align Snowflake source docs with implementation behavior Co-authored-by: tabVersion <14017336+tabVersion@users.noreply.github.com> --- ingestion/overview.mdx | 4 ++-- ingestion/sources/snowflake.mdx | 25 +++++++++++++++---------- 2 files changed, 17 insertions(+), 12 deletions(-) diff --git a/ingestion/overview.mdx b/ingestion/overview.mdx index e5f6532f..d37e3eb4 100644 --- a/ingestion/overview.mdx +++ b/ingestion/overview.mdx @@ -286,7 +286,7 @@ CREATE TABLE test_users ( | **Google Cloud Storage** | ❌ | ✅ | ⚠️ | Batch only; periodic via external tools | | **Azure Blob** | ❌ | ✅ | ⚠️ | Batch only; periodic via external tools | | **Apache Iceberg** | ❌ | ✅ | ⚠️ | Batch only; periodic via external tools | -| **Snowflake** | ❌ | ❌ | ✅ | Periodic refresh with `refresh_interval_sec` | +| **Snowflake** | ❌ | ✅ | ✅ | One-time batch load; periodic refresh when `refresh_interval_sec` is set | | **Datagen** | ✅ | ❌ | ❌ | Test data generation only | | **Direct INSERT** | ❌ | ✅ | ⚠️ | Manual insertion; periodic via external tools | | **Webhook** | ✅ | ✅ | ⚠️ | Push-based HTTP ingestion; best for SaaS webhooks + request validation/signatures | @@ -323,4 +323,4 @@ CREATE TABLE test_users ( - Monitor streaming lag for real-time sources to ensure data freshness. - Track batch job success and failure rates. - Set up alerts for data quality issues. -- Use RisingWave's system tables and dashboards for monitoring. \ No newline at end of file +- Use RisingWave's system tables and dashboards for monitoring. diff --git a/ingestion/sources/snowflake.mdx b/ingestion/sources/snowflake.mdx index 7341b4ee..65af7627 100644 --- a/ingestion/sources/snowflake.mdx +++ b/ingestion/sources/snowflake.mdx @@ -34,34 +34,37 @@ CREATE TABLE table_name ( adbc_snowflake.database = 'database_name', adbc_snowflake.schema = 'schema_name', adbc_snowflake.warehouse = 'warehouse_name', - adbc_snowflake.table = 'table_name' + adbc_snowflake.table = 'source_table_in_snowflake' ); ``` **Automatic Schema Inference** -Column definitions are automatically inferred from the Snowflake table and should not be manually specified in the `CREATE TABLE` statement. However, you must specify the primary key if your table requires one. +Column definitions are automatically inferred from the Snowflake table and should not be manually specified in the `CREATE TABLE` statement. You must specify a primary key when creating a RisingWave table that sources from Snowflake. ## Parameters -All parameters are required unless specified otherwise. +Unless specified otherwise, parameters are required. | Parameter | Description | | :--------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | connector | Must be `adbc_snowflake`. | | refresh_mode | Must be `FULL_RELOAD`. The entire table is re-read on each refresh. | -| refresh_interval_sec | The refresh interval in seconds. Determines how frequently data is fetched from Snowflake. | +| refresh_interval_sec | **Optional**. The refresh interval in seconds. Determines how frequently data is fetched from Snowflake. If omitted, data is loaded once at table creation and then refreshed only when you run [`REFRESH TABLE`](/sql/commands/sql-refresh-table). | | adbc_snowflake.account | The Snowflake account identifier (e.g., `myaccount.us-east-1` or `myaccount`). | | adbc_snowflake.username | The Snowflake username for authentication. | -| adbc_snowflake.password | **Optional**. The password for username/password authentication. Required if using the default `auth_snowflake` authentication type. | +| adbc_snowflake.password | **Optional**. The password for username/password authentication. Required when using username/password authentication (the default auth type). | | adbc_snowflake.database | The name of the Snowflake database. | | adbc_snowflake.schema | The Snowflake schema containing the table. | | adbc_snowflake.warehouse | The Snowflake warehouse to use for queries. | | adbc_snowflake.table | The name of the Snowflake table to ingest. | | adbc_snowflake.auth_type | **Optional**. The authentication method. Default is `auth_snowflake` (username/password). Other options: `auth_oauth`, `auth_jwt`, `auth_ext_browser`, `auth_okta`, `auth_mfa`, `auth_pat`, `auth_wif`. | -| adbc_snowflake.jwt_private_key_path | Required when `adbc_snowflake.auth_type` is `auth_jwt`. Local file path on the RisingWave server to the JWT private key file (e.g., `/path/to/key.pem`). | +| adbc_snowflake.auth_token | **Optional**. OAuth token for authentication. Required when `adbc_snowflake.auth_type` is `auth_oauth`. | +| adbc_snowflake.jwt_private_key_path | **Optional**. Local file path on the RisingWave server to the JWT private key file (e.g., `/path/to/key.pem`). For `auth_jwt`, provide this or `adbc_snowflake.jwt_private_key_pkcs8_value` (provide exactly one of the two). | +| adbc_snowflake.jwt_private_key_pkcs8_value | **Optional**. Inline PKCS#8 private key value for JWT authentication. For `auth_jwt`, provide this or `adbc_snowflake.jwt_private_key_path` (provide exactly one of the two). | +| adbc_snowflake.jwt_private_key_pkcs8_password | **Optional**. Password for `adbc_snowflake.jwt_private_key_pkcs8_value`, if the private key is encrypted. | ## Authentication methods @@ -121,7 +124,8 @@ CREATE TABLE my_snowflake_table ( adbc_snowflake.schema = 'PUBLIC', adbc_snowflake.warehouse = 'COMPUTE_WH', adbc_snowflake.table = 'ORDERS', - adbc_snowflake.auth_type = 'auth_oauth' + adbc_snowflake.auth_type = 'auth_oauth', + adbc_snowflake.auth_token = 'your_oauth_token' ); ``` @@ -196,14 +200,15 @@ GROUP BY DATE_TRUNC('day', order_date); ## Limitations and requirements -* **Refresh mode**: Only `FULL_RELOAD` mode is supported. The entire table is re-read on each refresh interval. +* **Refresh mode**: Only `FULL_RELOAD` mode is supported. If `refresh_interval_sec` is set, RisingWave refreshes on that schedule. If it is omitted, refresh manually with [`REFRESH TABLE`](/sql/commands/sql-refresh-table). +* **Primary key**: You must define a primary key when creating a refreshable Snowflake table. * **Schema inference**: Column definitions are automatically inferred from the Snowflake table. Do not manually specify columns in the `CREATE TABLE` statement. * **Feature flag**: The Snowflake connector requires the `source-adbc_snowflake` feature to be enabled at compile time. This is enabled by default in official RisingWave builds. -* **Consistent snapshots**: The connector uses Snowflake's time travel feature to ensure all data is read from the same point in time during a refresh. +* **Snapshot consistency**: The connector uses Snowflake's time travel feature when available to read data from a consistent point in time; when unavailable (for example, due to Snowflake retention settings), it falls back to reading current table data. * **Performance**: For large tables, consider the refresh interval carefully to balance data freshness with query costs in Snowflake. ## What's next? * [Sink data to Snowflake](/integrations/destinations/snowflake) - Learn how to write data from RisingWave back to Snowflake * [Work with Snowflake and Iceberg](/iceberg/integ-snowflake) - Integrate Snowflake with Apache Iceberg catalogs -* [Data formats and encoding options](/ingestion/formats-and-encoding-options) - Understand supported data formats +* [`REFRESH TABLE`](/sql/commands/sql-refresh-table) - Manually refresh a Snowflake table when `refresh_interval_sec` is not set From c919d5450e915a39854c7be81f04fe61c29bd136 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 26 Feb 2026 09:46:19 +0000 Subject: [PATCH 6/6] Clarify JWT auth_type note in Snowflake source docs Co-authored-by: tabVersion <14017336+tabVersion@users.noreply.github.com> --- ingestion/sources/snowflake.mdx | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/ingestion/sources/snowflake.mdx b/ingestion/sources/snowflake.mdx index 65af7627..0833bc7c 100644 --- a/ingestion/sources/snowflake.mdx +++ b/ingestion/sources/snowflake.mdx @@ -41,7 +41,7 @@ CREATE TABLE table_name ( **Automatic Schema Inference** -Column definitions are automatically inferred from the Snowflake table and should not be manually specified in the `CREATE TABLE` statement. You must specify a primary key when creating a RisingWave table that sources from Snowflake. +Column definitions are automatically inferred from the Snowflake table and should not be manually specified in the `CREATE TABLE` statement. You must specify a primary key when creating a Snowflake table in RisingWave. ## Parameters @@ -52,7 +52,7 @@ Unless specified otherwise, parameters are required. | :--------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | connector | Must be `adbc_snowflake`. | | refresh_mode | Must be `FULL_RELOAD`. The entire table is re-read on each refresh. | -| refresh_interval_sec | **Optional**. The refresh interval in seconds. Determines how frequently data is fetched from Snowflake. If omitted, data is loaded once at table creation and then refreshed only when you run [`REFRESH TABLE`](/sql/commands/sql-refresh-table). | +| refresh_interval_sec | **Optional**. The refresh interval in seconds. Determines how frequently data is fetched from Snowflake. | | adbc_snowflake.account | The Snowflake account identifier (e.g., `myaccount.us-east-1` or `myaccount`). | | adbc_snowflake.username | The Snowflake username for authentication. | | adbc_snowflake.password | **Optional**. The password for username/password authentication. Required when using username/password authentication (the default auth type). | @@ -62,10 +62,15 @@ Unless specified otherwise, parameters are required. | adbc_snowflake.table | The name of the Snowflake table to ingest. | | adbc_snowflake.auth_type | **Optional**. The authentication method. Default is `auth_snowflake` (username/password). Other options: `auth_oauth`, `auth_jwt`, `auth_ext_browser`, `auth_okta`, `auth_mfa`, `auth_pat`, `auth_wif`. | | adbc_snowflake.auth_token | **Optional**. OAuth token for authentication. Required when `adbc_snowflake.auth_type` is `auth_oauth`. | -| adbc_snowflake.jwt_private_key_path | **Optional**. Local file path on the RisingWave server to the JWT private key file (e.g., `/path/to/key.pem`). For `auth_jwt`, provide this or `adbc_snowflake.jwt_private_key_pkcs8_value` (provide exactly one of the two). | -| adbc_snowflake.jwt_private_key_pkcs8_value | **Optional**. Inline PKCS#8 private key value for JWT authentication. For `auth_jwt`, provide this or `adbc_snowflake.jwt_private_key_path` (provide exactly one of the two). | +| adbc_snowflake.jwt_private_key_path | **Optional**. Local file path on the RisingWave server to the JWT private key file (e.g., `/path/to/key.pem`). | +| adbc_snowflake.jwt_private_key_pkcs8_value | **Optional**. Inline PKCS#8 private key value for JWT authentication. | | adbc_snowflake.jwt_private_key_pkcs8_password | **Optional**. Password for `adbc_snowflake.jwt_private_key_pkcs8_value`, if the private key is encrypted. | + +- If `refresh_interval_sec` is omitted, data is loaded once at table creation, and subsequent refreshes require manually running [`REFRESH TABLE`](/sql/commands/sql-refresh-table). +- When using JWT authentication (set `adbc_snowflake.auth_type` to `auth_jwt`), provide exactly one of `adbc_snowflake.jwt_private_key_path` and `adbc_snowflake.jwt_private_key_pkcs8_value`. + + ## Authentication methods The Snowflake connector supports multiple authentication methods: @@ -201,7 +206,7 @@ GROUP BY DATE_TRUNC('day', order_date); ## Limitations and requirements * **Refresh mode**: Only `FULL_RELOAD` mode is supported. If `refresh_interval_sec` is set, RisingWave refreshes on that schedule. If it is omitted, refresh manually with [`REFRESH TABLE`](/sql/commands/sql-refresh-table). -* **Primary key**: You must define a primary key when creating a refreshable Snowflake table. +* **Primary key**: You must define a primary key when creating a Snowflake table in RisingWave. * **Schema inference**: Column definitions are automatically inferred from the Snowflake table. Do not manually specify columns in the `CREATE TABLE` statement. * **Feature flag**: The Snowflake connector requires the `source-adbc_snowflake` feature to be enabled at compile time. This is enabled by default in official RisingWave builds. * **Snapshot consistency**: The connector uses Snowflake's time travel feature when available to read data from a consistent point in time; when unavailable (for example, due to Snowflake retention settings), it falls back to reading current table data.