Skip to content

Commit 5112ebf

Browse files
committed
Update
1 parent 7bc7554 commit 5112ebf

File tree

4 files changed

+34
-25
lines changed

4 files changed

+34
-25
lines changed

articles/data-factory/connector-hive.md

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ ms.author: jianleishen
1616
This article outlines how to use the Copy Activity in an Azure Data Factory or Synapse Analytics pipeline to copy data from Hive. It builds on the [copy activity overview](copy-activity-overview.md) article that presents a general overview of copy activity.
1717

1818
> [!IMPORTANT]
19-
> The Hive connector version 2.0 (Preview) provides improved native Hive support. If you are using the Hive connector version 1.0 in your solution, please [upgrade your Hive connector](#upgrade-the-hive-connector) before **September 30, 2025**. Refer to this [section](#differences-between-hive-version-20-and-version-10) for details on the difference between version 2.0 (Preview) and version 1.0.
19+
> The Hive connector version 2.0 provides improved native Hive support. If you are using the Hive connector version 1.0 in your solution, please [upgrade your Hive connector](#upgrade-the-hive-connector) before **September 30, 2025**. Refer to this [section](#differences-between-hive-version-20-and-version-10) for details on the difference between version 2.0 and version 1.0.
2020
2121
## Supported capabilities
2222

@@ -40,6 +40,10 @@ The connector supports the Windows versions in this [article](create-self-hosted
4040

4141
[!INCLUDE [data-factory-v2-integration-runtime-requirements](includes/data-factory-v2-integration-runtime-requirements.md)]
4242

43+
> [!NOTE]
44+
> For Hive connector version 2.0, if you use the self-hosted integration runtime, its version should be 5.54 or above.
45+
46+
4347
## Getting started
4448

4549
[!INCLUDE [data-factory-v2-connector-get-started](includes/data-factory-v2-connector-get-started.md)]
@@ -74,14 +78,13 @@ The following sections provide details about properties that are used to define
7478

7579
## Linked service properties
7680

77-
The Hive connector now supports version 2.0 (Preview). Refer to this [section](#upgrade-the-hive-connector) to upgrade your Hive connector version from version 1.0. For the property details, see the corresponding sections.
81+
The Hive connector now supports version 2.0. Refer to this [section](#upgrade-the-hive-connector) to upgrade your Hive connector version from version 1.0. For the property details, see the corresponding sections.
7882

79-
- [Version 2.0 (Preview)](#version-20)
83+
- [Version 2.0](#version-20)
8084
- [Version 1.0](#version-10)
8185

82-
### <a name="version-20"></a> Version 2.0 (Preview)
83-
84-
The Hive linked service supports the following properties when apply version 2.0 (Preview):
86+
### <a name="version-20"></a> Version 2.0
87+
The Hive linked service supports the following properties when apply version 2.0:
8588

8689
| Property | Description | Required |
8790
|:--- |:--- |:--- |
@@ -98,7 +101,7 @@ The Hive linked service supports the following properties when apply version 2.0
98101
| enableSsl | Specifies whether the connections to the server are encrypted using TLS. The default value is true. | No |
99102
| enableServerCertificateValidation | Specify whether to enable server SSL certificate validation when you connect. Always use System Trust Store. The default value is true. | No |
100103
| storageReference | A reference to the linked service of the storage account used for staging data in mapping data flow. This is required only when using the Hive linked service in mapping data flow. | No |
101-
| connectVia | The [Integration Runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. |No |
104+
| connectVia | The [Integration Runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. If you use the self-hosted integration runtime, its version should be 5.54 or above. |No |
102105

103106
**Example:**
104107

@@ -306,7 +309,7 @@ source(
306309

307310
When you copy data from and to Hive, the following interim data type mappings are used within the service. To learn about how the copy activity maps the source schema and data type to the sink, see [Schema and data type mappings](copy-activity-schema-and-type-mapping.md).
308311

309-
| Hive data type | Interim service data type (for version 2.0 (Preview)) | Interim service data type (for version 1.0) |
312+
| Hive data type | Interim service data type (for version 2.0) | Interim service data type (for version 1.0) |
310313
|:--- |:--- |:--- |
311314
| TINYINT | Sbyte | Int16 |
312315
| SMALLINT | Int16 | Int16 |
@@ -338,13 +341,15 @@ The following table shows the release stage and change logs for different versio
338341
| Version | Release stage | Change log |
339342
| :----------- | :------- | :------- |
340343
| Version 1.0 | End of support announced | / |
341-
| Version 2.0 (Preview) | GA version available | • Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is not supported. <br><br>• HiveServer1 and HiveThriftServer are not supported for `ServerType`. <br><br>• Username authentication type is not supported. SASL transport protocol only supports UsernameAndPassword authentication type. Binary transport protocol only supports Anonymous authentication type. <br><br>• `serviceDiscoveryMode`, `zooKeeperNameSpace` and `useNativeQuery` are not supported. <br><br>• The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.<br>`enableServerCertificateValidation` is supported. <br><br>• TINYINT is read as SByte data type. <br><br>• TIMESTAMP is read as DateTimeOffset data type. |
344+
| Version 2.0 | GA version available | • Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is not supported. <br><br>• HiveServer1 and HiveThriftServer are not supported for `ServerType`. <br><br>• Username authentication type is not supported. SASL transport protocol only supports UsernameAndPassword authentication type. Binary transport protocol only supports Anonymous authentication type. <br><br>• `serviceDiscoveryMode`, `zooKeeperNameSpace` and `useNativeQuery` are not supported. <br><br>• The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.<br>`enableServerCertificateValidation` is supported. <br><br>• TINYINT is read as SByte data type. <br><br>• TIMESTAMP is read as DateTimeOffset data type. <br><br>• The self-hosted integration runtime version should be 5.54 or above. |
345+
346+
### <a name="upgrade-the-hive-connector"></a> Upgrade the Hive connector from version 1.0 to version 2.0
342347

343-
### <a name="upgrade-the-hive-connector"></a> Upgrade the Hive connector from version 1.0 to version 2.0 (Preview)
348+
1. In **Edit linked service** page, select version 2.0 and configure the linked service by referring to [Linked service properties version 2.0](#version-20).
344349

345-
1. In **Edit linked service** page, select version 2.0 (Preview) and configure the linked service by referring to [Linked service properties version 2.0 (Preview)](#version-20).
350+
2. The data type mapping for the Hive linked service version 2.0 is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Hive](#data-type-mapping-for-hive).
346351

347-
2. The data type mapping for the Hive linked service version 2.0 (Preview) is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Hive](#data-type-mapping-for-hive).
352+
3. If you use the version 2.0, your self-hosted integration runtime version should be 5.54 or above.
348353

349354
## Related content
350355
For a list of data stores supported as sources and sinks by the copy activity, see [supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).

articles/data-factory/connector-impala.md

Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ ms.author: jianleishen
1616
This article outlines how to use Copy Activity in an Azure Data Factory or Synapse Analytics pipeline to copy data from Impala. It builds on the [Copy Activity overview](copy-activity-overview.md) article that presents a general overview of the copy activity.
1717

1818
> [!IMPORTANT]
19-
> The Impala connector version 2.0 (Preview) provides improved native Impala support. If you are using the Impala connector version 1.0 in your solution, please [upgrade your Impala connector](#upgrade-the-impala-connector) before **September 30, 2025**. Refer to this [section](#differences-between-impala-version-20-and-version-10) for details on the difference between version 2.0 (Preview) and version 1.0.
19+
> The Impala connector version 2.0 provides improved native Impala support. If you are using the Impala connector version 1.0 in your solution, please [upgrade your Impala connector](#upgrade-the-impala-connector) before **September 30, 2025**. Refer to this [section](#differences-between-impala-version-20-and-version-10) for details on the difference between version 2.0 and version 1.0.
2020
2121
## Supported capabilities
2222

@@ -37,6 +37,9 @@ The service provides a built-in driver to enable connectivity. Therefore, you do
3737

3838
[!INCLUDE [data-factory-v2-integration-runtime-requirements](includes/data-factory-v2-integration-runtime-requirements.md)]
3939

40+
> [!NOTE]
41+
> For Impala connector version 2.0, if you use the self-hosted integration runtime, its version should be 5.55 or above.
42+
4043
## Get started
4144

4245
[!INCLUDE [data-factory-v2-connector-get-started](includes/data-factory-v2-connector-get-started.md)]
@@ -70,14 +73,14 @@ The following sections provide details about properties that are used to define
7073

7174
## Linked service properties
7275

73-
The Impala connector now supports version 2.0 (Preview). Refer to this [section](#upgrade-the-impala-connector) to upgrade your Impala connector version from version 1.0. For the property details, see the corresponding sections.
76+
The Impala connector now supports version 2.0. Refer to this [section](#upgrade-the-impala-connector) to upgrade your Impala connector version from version 1.0. For the property details, see the corresponding sections.
7477

75-
- [Version 2.0 (Preview)](#version-20)
78+
- [Version 2.0](#version-20)
7679
- [Version 1.0](#version-10)
7780

78-
### <a name="version-20"></a> Version 2.0 (Preview)
81+
### <a name="version-20"></a> Version 2.0
7982

80-
The Impala linked service supports the following properties when apply version 2.0 (Preview):
83+
The Impala linked service supports the following properties when apply version 2.0:
8184

8285
| Property | Description | Required |
8386
|:--- |:--- |:--- |
@@ -91,7 +94,7 @@ The Impala linked service supports the following properties when apply version 2
9194
| password | The password that corresponds to the user name when you use UsernameAndPassword. Mark this field as a SecureString to store it securely, or [reference a secret stored in Azure Key Vault](store-credentials-in-key-vault.md). | No |
9295
| enableSsl | Specifies whether the connections to the server are encrypted by using TLS. The default value is true. | No |
9396
| enableServerCertificateValidation | Specify whether to enable server SSL certificate validation when you connect. Always use System Trust Store. The default value is true. | No |
94-
| connectVia | The [integration runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. |No |
97+
| connectVia | The [integration runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. If you use the self-hosted integration runtime, its version should be 5.55 or above. |No |
9598

9699
**Example:**
97100

@@ -247,7 +250,7 @@ To copy data from Impala, set the source type in the copy activity to **ImpalaSo
247250

248251
When you copy data from and to Impala, the following interim data type mappings are used within the service. To learn about how the copy activity maps the source schema and data type to the sink, see [Schema and data type mappings](copy-activity-schema-and-type-mapping.md).
249252

250-
| Impala data type | Interim service data type (for version 2.0 (Preview)) | Interim service data type (for version 1.0) |
253+
| Impala data type | Interim service data type (for version 2.0) | Interim service data type (for version 1.0) |
251254
|:--- |:--- |:--- |
252255
| ARRAY | String | String |
253256
| BIGINT | Int64 | Int64 |
@@ -277,14 +280,15 @@ The following table shows the release stage and change logs for different versio
277280
| Version | Release stage | Change log |
278281
| :----------- | :------- | :------- |
279282
| Version 1.0 | End of support announced | / |
280-
| Version 2.0 (Preview) | GA version available | • SASLUsername authentication type is not supported. <br><br>• The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.<br>`enableServerCertificateValidation` is supported. <br><br>• TIMESTAMP is read as DateTimeOffset data type. <br><br>• TINYINT is read as SByte data type. |
283+
| Version 2.0 | GA version available | • SASLUsername authentication type is not supported. <br><br>• The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.<br>`enableServerCertificateValidation` is supported. <br><br>• TIMESTAMP is read as DateTimeOffset data type. <br><br>• TINYINT is read as SByte data type. <br><br>• The self-hosted integration runtime version should be 5.55 or above. |
281284

282-
### <a name="upgrade-the-impala-connector"></a> Upgrade the Impala connector from version 1.0 to version 2.0 (Preview)
285+
### <a name="upgrade-the-impala-connector"></a> Upgrade the Impala connector from version 1.0 to version 2.0
283286

284-
1. In **Edit linked service** page, select version 2.0 (Preview) and configure the linked service by referring to [Linked service properties version 2.0 (Preview)](#version-20).
287+
1. In **Edit linked service** page, select version 2.0 and configure the linked service by referring to [Linked service properties version 2.0](#version-20).
285288

286-
2. The data type mapping for the Impala linked service version 2.0 (Preview) is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Impala](#data-type-mapping-for-impala).
289+
2. The data type mapping for the Impala linked service version 2.0 is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Impala](#data-type-mapping-for-impala).
287290

291+
3. If you use the version 2.0, your self-hosted integration runtime version should be 5.55 or above.
288292

289293
## Related content
290294
For a list of data stores supported as sources and sinks by the copy activity, see [Supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).

articles/data-factory/connector-snowflake.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -715,7 +715,7 @@ The following table shows the release stage and change logs for different versio
715715
| :----------- | :------- |:------- |
716716
| Snowflake V1 | End of support announced | / |
717717
| Snowflake V2 (version 1.0) | GA version available | • Add support for Key pair authentication.<br><br>• The `accountIdentifier`, `warehouse`, `database`, `schema` and `role` properties are used to establish a connection instead of `connectionstring` property.<br><br>• Add support for Decimal in Lookup activity. The NUMBER type, as defined in Snowflake, will be displayed as a string in Lookup activity. If you want to covert it to numeric type in V2, you can use the pipeline parameter with [int function](control-flow-expression-language-functions.md#int) or [float function](control-flow-expression-language-functions.md#float). For example, `int(activity('lookup').output.firstRow.VALUE)`, `float(activity('lookup').output.firstRow.VALUE)`<br><br>• timestamp data type in Snowflake is read as DateTimeOffset data type in Lookup and Script activity. If you still need to use the Datetime value as a parameter in your pipeline after upgrading to V2, you can convert DateTimeOffset type to DateTime type by using [formatDateTime function](control-flow-expression-language-functions.md#formatdatetime) (recommended) or [concat function](control-flow-expression-language-functions.md#concat). For example: `formatDateTime(activity('lookup').output.firstRow.DATETIMETYPE)`, `concat(substring(activity('lookup').output.firstRow.DATETIMETYPE, 0, 19), 'Z')` <br><br>• NUMBER (p,0) is read as Decimal data type.<br><br>• TIMESTAMP_LTZ, TIMESTAMP_NTZ and TIMESTAMP_TZ is read as DateTimeOffset data type.<br><br>• Script parameters are not supported in Script activity. As an alternative, utilize dynamic expressions for script parameters. For more information, see [Expressions and functions in Azure Data Factory and Azure Synapse Analytics](control-flow-expression-language-functions.md).<br><br>• Multiple SQL statements execution in Script activity is not supported. |
718-
| Snowflake V2 (version 1.1) | Preview version available | • Add support for script parameters.<br><br>• Add support for multiple statement execution in Script activity. |
718+
| Snowflake V2 (version 1.1) | Preview version available | • Add support for script parameters.<br><br>• Add support for multiple statement execution in Script activity. <br><br>• Add support for storage integration in Copy activity. |
719719

720720
### <a name="upgrade-the-snowflake-linked-service"></a> Upgrade the Snowflake connector from V1 to V2
721721

articles/data-factory/connector-spark.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -280,7 +280,7 @@ The Spark connector version 2.0 offers new functionalities and is compatible wit
280280
| Binary and SASL are not supported for `thriftTransportProtocl`. | Support Binary and SASL for `thriftTransportProtocl`. |
281281
| Username authentication type is not supported. | Support Username authentication type. |
282282
| The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported. <br><br> `enableServerCertificateValidation` is supported. | The default value of `enableSSL` is false. Additionally, support `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert`. <br><br>`enableServerCertificateValidation` is not supported. |
283-
| The following mappings are used from Spark data types to interim service data types used by the service internally.<br><br>TimestampType -> DateTimeOffset <br>YearMonthIntervalType -> String<br>DayTimeIntervalType -> String | The following mappings are used from Spark data types to interim service data types used by the service internally.<br><br>TimestampType -> DateTime<br>Other mappings supported by version 2.0 (Preview) listed left are not supported by version 1.0. |
283+
| The following mappings are used from Spark data types to interim service data types used by the service internally.<br><br>TimestampType -> DateTimeOffset <br>YearMonthIntervalType -> String<br>DayTimeIntervalType -> String | The following mappings are used from Spark data types to interim service data types used by the service internally.<br><br>TimestampType -> DateTime<br>Other mappings supported by version 2.0 listed left are not supported by version 1.0. |
284284

285285
## Related content
286286
For a list of data stores supported as sources and sinks by the copy activity, see [supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).

0 commit comments

Comments
 (0)