You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/connector-hive.md
+17-12Lines changed: 17 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ ms.author: jianleishen
16
16
This article outlines how to use the Copy Activity in an Azure Data Factory or Synapse Analytics pipeline to copy data from Hive. It builds on the [copy activity overview](copy-activity-overview.md) article that presents a general overview of copy activity.
17
17
18
18
> [!IMPORTANT]
19
-
> The Hive connector version 2.0 (Preview) provides improved native Hive support. If you are using the Hive connector version 1.0 in your solution, please [upgrade your Hive connector](#upgrade-the-hive-connector) before **September 30, 2025**. Refer to this [section](#differences-between-hive-version-20-and-version-10) for details on the difference between version 2.0 (Preview) and version 1.0.
19
+
> The Hive connector version 2.0 provides improved native Hive support. If you are using the Hive connector version 1.0 in your solution, please [upgrade your Hive connector](#upgrade-the-hive-connector) before **September 30, 2025**. Refer to this [section](#differences-between-hive-version-20-and-version-10) for details on the difference between version 2.0 and version 1.0.
20
20
21
21
## Supported capabilities
22
22
@@ -40,6 +40,10 @@ The connector supports the Windows versions in this [article](create-self-hosted
@@ -74,14 +78,13 @@ The following sections provide details about properties that are used to define
74
78
75
79
## Linked service properties
76
80
77
-
The Hive connector now supports version 2.0 (Preview). Refer to this [section](#upgrade-the-hive-connector) to upgrade your Hive connector version from version 1.0. For the property details, see the corresponding sections.
81
+
The Hive connector now supports version 2.0. Refer to this [section](#upgrade-the-hive-connector) to upgrade your Hive connector version from version 1.0. For the property details, see the corresponding sections.
78
82
79
-
-[Version 2.0 (Preview)](#version-20)
83
+
-[Version 2.0](#version-20)
80
84
-[Version 1.0](#version-10)
81
85
82
-
### <aname="version-20"></a> Version 2.0 (Preview)
83
-
84
-
The Hive linked service supports the following properties when apply version 2.0 (Preview):
86
+
### <aname="version-20"></a> Version 2.0
87
+
The Hive linked service supports the following properties when apply version 2.0:
85
88
86
89
| Property | Description | Required |
87
90
|:--- |:--- |:--- |
@@ -98,7 +101,7 @@ The Hive linked service supports the following properties when apply version 2.0
98
101
| enableSsl | Specifies whether the connections to the server are encrypted using TLS. The default value is true. | No |
99
102
| enableServerCertificateValidation | Specify whether to enable server SSL certificate validation when you connect. Always use System Trust Store. The default value is true. | No |
100
103
| storageReference | A reference to the linked service of the storage account used for staging data in mapping data flow. This is required only when using the Hive linked service in mapping data flow. | No |
101
-
| connectVia | The [Integration Runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. |No |
104
+
| connectVia | The [Integration Runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. If you use the self-hosted integration runtime, its version should be 5.54 or above. |No |
102
105
103
106
**Example:**
104
107
@@ -306,7 +309,7 @@ source(
306
309
307
310
When you copy data from and to Hive, the following interim data type mappings are used within the service. To learn about how the copy activity maps the source schema and data type to the sink, see [Schema and data type mappings](copy-activity-schema-and-type-mapping.md).
308
311
309
-
| Hive data type | Interim service data type (for version 2.0 (Preview)) | Interim service data type (for version 1.0) |
312
+
| Hive data type | Interim service data type (for version 2.0) | Interim service data type (for version 1.0) |
310
313
|:--- |:--- |:--- |
311
314
| TINYINT | Sbyte | Int16 |
312
315
| SMALLINT | Int16 | Int16 |
@@ -338,13 +341,15 @@ The following table shows the release stage and change logs for different versio
338
341
| Version | Release stage | Change log |
339
342
| :----------- | :------- | :------- |
340
343
| Version 1.0 | End of support announced | / |
341
-
| Version 2.0 (Preview) | GA version available | • Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is not supported. <br><br>• HiveServer1 and HiveThriftServer are not supported for `ServerType`. <br><br>• Username authentication type is not supported. SASL transport protocol only supports UsernameAndPassword authentication type. Binary transport protocol only supports Anonymous authentication type. <br><br>• `serviceDiscoveryMode`, `zooKeeperNameSpace` and `useNativeQuery` are not supported. <br><br>• The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.<br>`enableServerCertificateValidation` is supported. <br><br>• TINYINT is read as SByte data type. <br><br>• TIMESTAMP is read as DateTimeOffset data type. |
344
+
| Version 2.0 | GA version available | • Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is not supported. <br><br>• HiveServer1 and HiveThriftServer are not supported for `ServerType`. <br><br>• Username authentication type is not supported. SASL transport protocol only supports UsernameAndPassword authentication type. Binary transport protocol only supports Anonymous authentication type. <br><br>• `serviceDiscoveryMode`, `zooKeeperNameSpace` and `useNativeQuery` are not supported. <br><br>• The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.<br>`enableServerCertificateValidation` is supported. <br><br>• TINYINT is read as SByte data type. <br><br>• TIMESTAMP is read as DateTimeOffset data type. <br><br>• The self-hosted integration runtime version should be 5.54 or above. |
345
+
346
+
### <aname="upgrade-the-hive-connector"></a> Upgrade the Hive connector from version 1.0 to version 2.0
342
347
343
-
### <aname="upgrade-the-hive-connector"></a> Upgrade the Hive connector from version 1.0 to version 2.0 (Preview)
348
+
1. In **Edit linked service** page, select version 2.0 and configure the linked service by referring to [Linked service properties version 2.0](#version-20).
344
349
345
-
1. In **Edit linked service** page, select version 2.0 (Preview) and configure the linked service by referring to [Linked service properties version 2.0 (Preview)](#version-20).
350
+
2. The data type mapping for the Hive linked service version 2.0 is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Hive](#data-type-mapping-for-hive).
346
351
347
-
2. The data type mapping for the Hive linked service version 2.0 (Preview) is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Hive](#data-type-mapping-for-hive).
352
+
3. If you use the version 2.0, your self-hosted integration runtime version should be 5.54 or above.
348
353
349
354
## Related content
350
355
For a list of data stores supported as sources and sinks by the copy activity, see [supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).
Copy file name to clipboardExpand all lines: articles/data-factory/connector-impala.md
+15-11Lines changed: 15 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ ms.author: jianleishen
16
16
This article outlines how to use Copy Activity in an Azure Data Factory or Synapse Analytics pipeline to copy data from Impala. It builds on the [Copy Activity overview](copy-activity-overview.md) article that presents a general overview of the copy activity.
17
17
18
18
> [!IMPORTANT]
19
-
> The Impala connector version 2.0 (Preview) provides improved native Impala support. If you are using the Impala connector version 1.0 in your solution, please [upgrade your Impala connector](#upgrade-the-impala-connector) before **September 30, 2025**. Refer to this [section](#differences-between-impala-version-20-and-version-10) for details on the difference between version 2.0 (Preview) and version 1.0.
19
+
> The Impala connector version 2.0 provides improved native Impala support. If you are using the Impala connector version 1.0 in your solution, please [upgrade your Impala connector](#upgrade-the-impala-connector) before **September 30, 2025**. Refer to this [section](#differences-between-impala-version-20-and-version-10) for details on the difference between version 2.0 and version 1.0.
20
20
21
21
## Supported capabilities
22
22
@@ -37,6 +37,9 @@ The service provides a built-in driver to enable connectivity. Therefore, you do
@@ -70,14 +73,14 @@ The following sections provide details about properties that are used to define
70
73
71
74
## Linked service properties
72
75
73
-
The Impala connector now supports version 2.0 (Preview). Refer to this [section](#upgrade-the-impala-connector) to upgrade your Impala connector version from version 1.0. For the property details, see the corresponding sections.
76
+
The Impala connector now supports version 2.0. Refer to this [section](#upgrade-the-impala-connector) to upgrade your Impala connector version from version 1.0. For the property details, see the corresponding sections.
74
77
75
-
-[Version 2.0 (Preview)](#version-20)
78
+
-[Version 2.0](#version-20)
76
79
-[Version 1.0](#version-10)
77
80
78
-
### <aname="version-20"></a> Version 2.0 (Preview)
81
+
### <aname="version-20"></a> Version 2.0
79
82
80
-
The Impala linked service supports the following properties when apply version 2.0 (Preview):
83
+
The Impala linked service supports the following properties when apply version 2.0:
81
84
82
85
| Property | Description | Required |
83
86
|:--- |:--- |:--- |
@@ -91,7 +94,7 @@ The Impala linked service supports the following properties when apply version 2
91
94
| password | The password that corresponds to the user name when you use UsernameAndPassword. Mark this field as a SecureString to store it securely, or [reference a secret stored in Azure Key Vault](store-credentials-in-key-vault.md). | No |
92
95
| enableSsl | Specifies whether the connections to the server are encrypted by using TLS. The default value is true. | No |
93
96
| enableServerCertificateValidation | Specify whether to enable server SSL certificate validation when you connect. Always use System Trust Store. The default value is true. | No |
94
-
| connectVia | The [integration runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. |No |
97
+
| connectVia | The [integration runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. If you use the self-hosted integration runtime, its version should be 5.55 or above. |No |
95
98
96
99
**Example:**
97
100
@@ -247,7 +250,7 @@ To copy data from Impala, set the source type in the copy activity to **ImpalaSo
247
250
248
251
When you copy data from and to Impala, the following interim data type mappings are used within the service. To learn about how the copy activity maps the source schema and data type to the sink, see [Schema and data type mappings](copy-activity-schema-and-type-mapping.md).
249
252
250
-
| Impala data type | Interim service data type (for version 2.0 (Preview)) | Interim service data type (for version 1.0) |
253
+
| Impala data type | Interim service data type (for version 2.0) | Interim service data type (for version 1.0) |
251
254
|:--- |:--- |:--- |
252
255
| ARRAY | String | String |
253
256
| BIGINT | Int64 | Int64 |
@@ -277,14 +280,15 @@ The following table shows the release stage and change logs for different versio
277
280
| Version | Release stage | Change log |
278
281
| :----------- | :------- | :------- |
279
282
| Version 1.0 | End of support announced | / |
280
-
| Version 2.0 (Preview) | GA version available | • SASLUsername authentication type is not supported. <br><br>• The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.<br>`enableServerCertificateValidation` is supported. <br><br>• TIMESTAMP is read as DateTimeOffset data type. <br><br>• TINYINT is read as SByte data type. |
283
+
| Version 2.0 | GA version available | • SASLUsername authentication type is not supported. <br><br>• The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.<br>`enableServerCertificateValidation` is supported. <br><br>• TIMESTAMP is read as DateTimeOffset data type. <br><br>• TINYINT is read as SByte data type. <br><br>• The self-hosted integration runtime version should be 5.55 or above. |
281
284
282
-
### <aname="upgrade-the-impala-connector"></a> Upgrade the Impala connector from version 1.0 to version 2.0 (Preview)
285
+
### <aname="upgrade-the-impala-connector"></a> Upgrade the Impala connector from version 1.0 to version 2.0
283
286
284
-
1. In **Edit linked service** page, select version 2.0 (Preview) and configure the linked service by referring to [Linked service properties version 2.0 (Preview)](#version-20).
287
+
1. In **Edit linked service** page, select version 2.0 and configure the linked service by referring to [Linked service properties version 2.0](#version-20).
285
288
286
-
2. The data type mapping for the Impala linked service version 2.0 (Preview) is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Impala](#data-type-mapping-for-impala).
289
+
2. The data type mapping for the Impala linked service version 2.0 is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Impala](#data-type-mapping-for-impala).
287
290
291
+
3. If you use the version 2.0, your self-hosted integration runtime version should be 5.55 or above.
288
292
289
293
## Related content
290
294
For a list of data stores supported as sources and sinks by the copy activity, see [Supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).
Copy file name to clipboardExpand all lines: articles/data-factory/connector-snowflake.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -715,7 +715,7 @@ The following table shows the release stage and change logs for different versio
715
715
| :----------- | :------- |:------- |
716
716
| Snowflake V1 | End of support announced | / |
717
717
| Snowflake V2 (version 1.0) | GA version available | • Add support for Key pair authentication.<br><br>• The `accountIdentifier`, `warehouse`, `database`, `schema` and `role` properties are used to establish a connection instead of `connectionstring` property.<br><br>• Add support for Decimal in Lookup activity. The NUMBER type, as defined in Snowflake, will be displayed as a string in Lookup activity. If you want to covert it to numeric type in V2, you can use the pipeline parameter with [int function](control-flow-expression-language-functions.md#int) or [float function](control-flow-expression-language-functions.md#float). For example, `int(activity('lookup').output.firstRow.VALUE)`, `float(activity('lookup').output.firstRow.VALUE)`<br><br>• timestamp data type in Snowflake is read as DateTimeOffset data type in Lookup and Script activity. If you still need to use the Datetime value as a parameter in your pipeline after upgrading to V2, you can convert DateTimeOffset type to DateTime type by using [formatDateTime function](control-flow-expression-language-functions.md#formatdatetime) (recommended) or [concat function](control-flow-expression-language-functions.md#concat). For example: `formatDateTime(activity('lookup').output.firstRow.DATETIMETYPE)`, `concat(substring(activity('lookup').output.firstRow.DATETIMETYPE, 0, 19), 'Z')` <br><br>• NUMBER (p,0) is read as Decimal data type.<br><br>• TIMESTAMP_LTZ, TIMESTAMP_NTZ and TIMESTAMP_TZ is read as DateTimeOffset data type.<br><br>• Script parameters are not supported in Script activity. As an alternative, utilize dynamic expressions for script parameters. For more information, see [Expressions and functions in Azure Data Factory and Azure Synapse Analytics](control-flow-expression-language-functions.md).<br><br>• Multiple SQL statements execution in Script activity is not supported. |
718
-
| Snowflake V2 (version 1.1) | Preview version available | • Add support for script parameters.<br><br>• Add support for multiple statement execution in Script activity. |
718
+
| Snowflake V2 (version 1.1) | Preview version available | • Add support for script parameters.<br><br>• Add support for multiple statement execution in Script activity. <br><br>• Add support for storage integration in Copy activity. |
719
719
720
720
### <aname="upgrade-the-snowflake-linked-service"></a> Upgrade the Snowflake connector from V1 to V2
Copy file name to clipboardExpand all lines: articles/data-factory/connector-spark.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -280,7 +280,7 @@ The Spark connector version 2.0 offers new functionalities and is compatible wit
280
280
| Binary and SASL are not supported for `thriftTransportProtocl`. | Support Binary and SASL for `thriftTransportProtocl`. |
281
281
| Username authentication type is not supported. | Support Username authentication type. |
282
282
| The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported. <br><br> `enableServerCertificateValidation` is supported. | The default value of `enableSSL` is false. Additionally, support `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert`. <br><br>`enableServerCertificateValidation` is not supported. |
283
-
| The following mappings are used from Spark data types to interim service data types used by the service internally.<br><br>TimestampType -> DateTimeOffset <br>YearMonthIntervalType -> String<br>DayTimeIntervalType -> String | The following mappings are used from Spark data types to interim service data types used by the service internally.<br><br>TimestampType -> DateTime<br>Other mappings supported by version 2.0 (Preview) listed left are not supported by version 1.0. |
283
+
| The following mappings are used from Spark data types to interim service data types used by the service internally.<br><br>TimestampType -> DateTimeOffset <br>YearMonthIntervalType -> String<br>DayTimeIntervalType -> String | The following mappings are used from Spark data types to interim service data types used by the service internally.<br><br>TimestampType -> DateTime<br>Other mappings supported by version 2.0 listed left are not supported by version 1.0. |
284
284
285
285
## Related content
286
286
For a list of data stores supported as sources and sinks by the copy activity, see [supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).
0 commit comments