Skip to content

Commit 66c5ae0

Browse files
prmerger-automator[bot]prmerger-automator[bot]
authored
Merge pull request #299888 from Clare-Zheng82/0514-Add_Snowflake_data_type_mapping
[Doc update] Add snowflake data type mapping
2 parents 8226806 + f80b434 commit 66c5ae0

File tree

2 files changed

+47
-3
lines changed

2 files changed

+47
-3
lines changed

‎articles/data-factory/connector-snowflake-legacy.md

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ author: jianleishen
77
ms.subservice: data-movement
88
ms.topic: conceptual
99
ms.custom: synapse
10-
ms.date: 05/07/2025
10+
ms.date: 05/15/2025
1111
---
1212

1313
# Copy and transform data in Snowflake V1 using Azure Data Factory or Azure Synapse Analytics
@@ -563,6 +563,28 @@ By setting the pipeline Logging Level to None, we exclude the transmission of in
563563
> [!NOTE]
564564
> We don’t support temporary tables in Snowflake, as they are local to the session or user who creates them, making them inaccessible to other sessions and prone to being overwritten as regular tables by Snowflake. While Snowflake offers transient tables as an alternative, which are accessible globally, they require manual deletion, contradicting our primary objective of using Temp tables which is to avoid any delete operations in source schema.
565565
566+
## Data type mapping for Snowflake V1
567+
568+
When you copy data from Snowflake, the following mappings are used from Snowflake data types to interim data types within the service internally. To learn about how the copy activity maps the source schema and data type to the sink, see [Schema and data type mappings](copy-activity-schema-and-type-mapping.md).
569+
570+
| Snowflake data type | Service interim data type |
571+
|--------------------|---------------------------------------------|
572+
| NUMBER (p,0) | Int64 |
573+
| NUMBER (p,s where s>0) | Decimal |
574+
| FLOAT | Double |
575+
| VARCHAR | String |
576+
| CHAR | String |
577+
| BINARY | Byte[] |
578+
| BOOLEAN | Boolean |
579+
| DATE | DateTime |
580+
| TIME | TimeSpan |
581+
| TIMESTAMP_LTZ | DateTime |
582+
| TIMESTAMP_NTZ | DateTime |
583+
| TIMESTAMP_TZ | DateTime |
584+
| VARIANT | String |
585+
| OBJECT | String |
586+
| ARRAY | String |
587+
566588
## Lookup activity properties
567589

568590
For more information about the properties, see [Lookup activity](control-flow-lookup-activity.md).

‎articles/data-factory/connector-snowflake.md

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ author: jianleishen
77
ms.subservice: data-movement
88
ms.topic: conceptual
99
ms.custom: synapse
10-
ms.date: 05/07/2025
10+
ms.date: 05/15/2025
1111
ai-usage: ai-assisted
1212
---
1313

@@ -681,6 +681,28 @@ By setting the pipeline Logging Level to None, we exclude the transmission of in
681681
> [!NOTE]
682682
> We don't support temporary tables in Snowflake, as they are local to the session or user who creates them, making them inaccessible to other sessions and prone to being overwritten as regular tables by Snowflake. While Snowflake offers transient tables as an alternative, which are accessible globally, they require manual deletion, contradicting our primary objective of using Temp tables which is to avoid any delete operations in source schema.
683683
684+
## Data type mapping for Snowflake V2
685+
686+
When you copy data from Snowflake, the following mappings are used from Snowflake data types to interim data types within the service internally. To learn about how the copy activity maps the source schema and data type to the sink, see [Schema and data type mappings](copy-activity-schema-and-type-mapping.md).
687+
688+
| Snowflake data type | Service interim data type |
689+
|--------------------|---------------------------------------------|
690+
| NUMBER (p,0) | Decimal |
691+
| NUMBER (p,s where s>0) | Decimal |
692+
| FLOAT | Double |
693+
| VARCHAR | String |
694+
| CHAR | String |
695+
| BINARY | Byte[] |
696+
| BOOLEAN | Boolean |
697+
| DATE | DateTime |
698+
| TIME | TimeSpan |
699+
| TIMESTAMP_LTZ | DateTimeOffset |
700+
| TIMESTAMP_NTZ | DateTimeOffset |
701+
| TIMESTAMP_TZ | DateTimeOffset |
702+
| VARIANT | String |
703+
| OBJECT | String |
704+
| ARRAY | String |
705+
684706
## Lookup activity properties
685707

686708
For more information about the properties, see [Lookup activity](control-flow-lookup-activity.md).
@@ -692,7 +714,7 @@ The following table shows the release stage and change logs for different versio
692714
| Version | Release stage | Change log |
693715
| :----------- | :------- |:------- |
694716
| Snowflake V1 | GA version available | / |
695-
| Snowflake V2 (version 1.0) | GA version available | • Add support for Key pair authentication.<br><br>• The `accountIdentifier`, `warehouse`, `database`, `schema` and `role` properties are used to establish a connection instead of `connectionstring` property.<br><br>• Add support for BigDecimal in Lookup activity. The NUMBER type, as defined in Snowflake, will be displayed as a string in Lookup activity. If you want to covert it to numeric type in V2, you can use the pipeline parameter with [int function](control-flow-expression-language-functions.md#int) or [float function](control-flow-expression-language-functions.md#float). For example, `int(activity('lookup').output.firstRow.VALUE)`, `float(activity('lookup').output.firstRow.VALUE)`<br><br>• timestamp data type in Snowflake is read as DateTimeOffset data type in Lookup and Script activity. If you still need to use the Datetime value as a parameter in your pipeline after upgrading to V2, you can convert DateTimeOffset type to DateTime type by using [formatDateTime function](control-flow-expression-language-functions.md#formatdatetime) (recommended) or [concat function](control-flow-expression-language-functions.md#concat). For example: `formatDateTime(activity('lookup').output.firstRow.DATETIMETYPE)`, `concat(substring(activity('lookup').output.firstRow.DATETIMETYPE, 0, 19), 'Z')`<br><br>• Script parameters are not supported in Script activity. As an alternative, utilize dynamic expressions for script parameters. For more information, see [Expressions and functions in Azure Data Factory and Azure Synapse Analytics](control-flow-expression-language-functions.md).<br><br>• Multiple SQL statements execution in Script activity is not supported. |
717+
| Snowflake V2 (version 1.0) | GA version available | • Add support for Key pair authentication.<br><br>• The `accountIdentifier`, `warehouse`, `database`, `schema` and `role` properties are used to establish a connection instead of `connectionstring` property.<br><br>• Add support for Decimal in Lookup activity. The NUMBER type, as defined in Snowflake, will be displayed as a string in Lookup activity. If you want to covert it to numeric type in V2, you can use the pipeline parameter with [int function](control-flow-expression-language-functions.md#int) or [float function](control-flow-expression-language-functions.md#float). For example, `int(activity('lookup').output.firstRow.VALUE)`, `float(activity('lookup').output.firstRow.VALUE)`<br><br>• timestamp data type in Snowflake is read as DateTimeOffset data type in Lookup and Script activity. If you still need to use the Datetime value as a parameter in your pipeline after upgrading to V2, you can convert DateTimeOffset type to DateTime type by using [formatDateTime function](control-flow-expression-language-functions.md#formatdatetime) (recommended) or [concat function](control-flow-expression-language-functions.md#concat). For example: `formatDateTime(activity('lookup').output.firstRow.DATETIMETYPE)`, `concat(substring(activity('lookup').output.firstRow.DATETIMETYPE, 0, 19), 'Z')` <br><br>• NUMBER (p,0) is read as Decimal data type.<br><br>• TIMESTAMP_LTZ, TIMESTAMP_NTZ and TIMESTAMP_TZ is read as DateTimeOffset data type.<br><br>• Script parameters are not supported in Script activity. As an alternative, utilize dynamic expressions for script parameters. For more information, see [Expressions and functions in Azure Data Factory and Azure Synapse Analytics](control-flow-expression-language-functions.md).<br><br>• Multiple SQL statements execution in Script activity is not supported. |
696718
| Snowflake V2 (version 1.1) | Preview version available | • Add support for script parameters.<br><br>• Add support for multiple statement execution in Script activity. |
697719

698720
### <a name="upgrade-the-snowflake-linked-service"></a> Upgrade the Snowflake connector from V1 to V2

0 commit comments

Comments
 (0)