Skip to content

Commit 9514657

Browse files
Merge pull request #301349 from Lucky-Wang16/0616-Update_lifecycle
Update lifecycle section for Amazon Redshift, Hive and Impala
2 parents ef355b3 + ec6dff2 commit 9514657

File tree

6 files changed

+57
-57
lines changed

6 files changed

+57
-57
lines changed

articles/data-factory/connector-amazon-redshift.md

Lines changed: 10 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ author: jianleishen
77
ms.subservice: data-movement
88
ms.custom: synapse
99
ms.topic: conceptual
10-
ms.date: 06/12/2025
10+
ms.date: 06/16/2025
1111
---
1212

1313
# Copy data from Amazon Redshift using Azure Data Factory or Synapse Analytics
@@ -289,24 +289,22 @@ When you copy data from Amazon Redshift, the following mappings apply from Amazo
289289

290290
To learn details about the properties, check [Lookup activity](control-flow-lookup-activity.md).
291291

292-
## Upgrade the Amazon Redshift connector
292+
## <a name="differences-between-amazon-redshift-connector-version-20-and-version-10"></a> Amazon Redshift connector lifecycle and upgrade
293293

294-
Here are steps that help you upgrade the Amazon Redshift connector:
294+
The following table shows the release stage and change logs for different versions of the Impala connector:
295+
296+
| Version | Release stage | Change log |
297+
| :----------- | :------- | :------- |
298+
| Version 1.0 | GA version available | / |
299+
| Version 2.0 (Preview) | Preview version available | • Only support the self-hosted integration runtime with version 5.54.0.0 or above. <br><br>• BOOLEAN is read as Boolean data type. |
300+
301+
### <a name="upgrade-the-amazon-redshift-connector"></a> Upgrade the Amazon Redshift connector from version 1.0 to version 2.0 (Preview)
295302

296303
1. In **Edit linked service** page, select version 2.0 (Preview) and configure the linked service by referring to [linked service properties](#linked-service-properties).
297304

298305
2. The data type mapping for the Amazon Redshift linked service version 2.0 (Preview) is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Amazon Redshift](#data-type-mapping-for-amazon-redshift).
299306

300307
3. Apply a self-hosted integration runtime with version 5.54.0.0 or above. Azure integration runtime is not supported by version 2.0 (Preview).
301308

302-
## <a name="differences-between-amazon-redshift-connector-version-20-and-version-10"></a>Differences between Amazon Redshift connector version 2.0 (Preview) and version 1.0
303-
304-
The Amazon Redshift connector version 2.0 (Preview) offers new functionalities and is compatible with most features of version 1.0. The following table shows the feature differences between version 2.0 (Preview) and version 1.0.
305-
306-
| Version 2.0 (Preview) | Version 1.0 |
307-
| :----------- | :------- |
308-
| Only support the self-hosted integration runtime with version 5.54.0.0 or above. | Support the Azure integration runtime and self-hosted integration runtime. |
309-
| The following mappings are used from Amazon Redshift data types to interim service data type.<br><br>BOOLEAN -> Boolean | The following mappings are used from Amazon Redshift data types to interim service data type.<br><br>BOOLEAN -> String |
310-
311309
## Related content
312310
For a list of data stores supported as sources and sinks by the copy activity, see [supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).

articles/data-factory/connector-hive.md

Lines changed: 21 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ author: jianleishen
66
ms.subservice: data-movement
77
ms.custom: synapse
88
ms.topic: conceptual
9-
ms.date: 05/27/2025
9+
ms.date: 06/16/2025
1010
ms.author: jianleishen
1111
---
1212

@@ -16,7 +16,7 @@ ms.author: jianleishen
1616
This article outlines how to use the Copy Activity in an Azure Data Factory or Synapse Analytics pipeline to copy data from Hive. It builds on the [copy activity overview](copy-activity-overview.md) article that presents a general overview of copy activity.
1717

1818
> [!IMPORTANT]
19-
> The Hive connector version 2.0 (Preview) provides improved native Hive support. If you are using the Hive connector version 1.0 in your solution, please [upgrade your Hive connector](#upgrade-the-hive-connector) before **September 30, 2025**. Refer to this [section](#differences-between-hive-version-20-and-version-10) for details on the difference between version 2.0 (Preview) and version 1.0.
19+
> The Hive connector version 2.0 provides improved native Hive support. If you are using the Hive connector version 1.0 in your solution, please [upgrade your Hive connector](#upgrade-the-hive-connector) before **September 30, 2025**. Refer to this [section](#differences-between-hive-version-20-and-version-10) for details on the difference between version 2.0 and version 1.0.
2020
2121
## Supported capabilities
2222

@@ -40,6 +40,9 @@ The connector supports the Windows versions in this [article](create-self-hosted
4040

4141
[!INCLUDE [data-factory-v2-integration-runtime-requirements](includes/data-factory-v2-integration-runtime-requirements.md)]
4242

43+
> [!NOTE]
44+
> Version 2.0 is supported with the self-hosted integration runtime version 5.54 or above.
45+
4346
## Getting started
4447

4548
[!INCLUDE [data-factory-v2-connector-get-started](includes/data-factory-v2-connector-get-started.md)]
@@ -74,14 +77,13 @@ The following sections provide details about properties that are used to define
7477

7578
## Linked service properties
7679

77-
The Hive connector now supports version 2.0 (Preview). Refer to this [section](#upgrade-the-hive-connector) to upgrade your Hive connector version from version 1.0. For the property details, see the corresponding sections.
80+
The Hive connector now supports version 2.0. Refer to this [section](#upgrade-the-hive-connector) to upgrade your Hive connector version from version 1.0. For the property details, see the corresponding sections.
7881

79-
- [Version 2.0 (Preview)](#version-20)
82+
- [Version 2.0](#version-20)
8083
- [Version 1.0](#version-10)
8184

82-
### <a name="version-20"></a> Version 2.0 (Preview)
83-
84-
The Hive linked service supports the following properties when apply version 2.0 (Preview):
85+
### <a name="version-20"></a> Version 2.0
86+
The Hive linked service supports the following properties when apply version 2.0:
8587

8688
| Property | Description | Required |
8789
|:--- |:--- |:--- |
@@ -98,7 +100,7 @@ The Hive linked service supports the following properties when apply version 2.0
98100
| enableSsl | Specifies whether the connections to the server are encrypted using TLS. The default value is true. | No |
99101
| enableServerCertificateValidation | Specify whether to enable server SSL certificate validation when you connect. Always use System Trust Store. The default value is true. | No |
100102
| storageReference | A reference to the linked service of the storage account used for staging data in mapping data flow. This is required only when using the Hive linked service in mapping data flow. | No |
101-
| connectVia | The [Integration Runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. |No |
103+
| connectVia | The [Integration Runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. You can use the self-hosted integration runtime and its version should be 5.54 or above. |No |
102104

103105
**Example:**
104106

@@ -306,7 +308,7 @@ source(
306308

307309
When you copy data from and to Hive, the following interim data type mappings are used within the service. To learn about how the copy activity maps the source schema and data type to the sink, see [Schema and data type mappings](copy-activity-schema-and-type-mapping.md).
308310

309-
| Hive data type | Interim service data type (for version 2.0 (Preview)) | Interim service data type (for version 1.0) |
311+
| Hive data type | Interim service data type (for version 2.0) | Interim service data type (for version 1.0) |
310312
|:--- |:--- |:--- |
311313
| TINYINT | Sbyte | Int16 |
312314
| SMALLINT | Int16 | Int16 |
@@ -330,27 +332,23 @@ When you copy data from and to Hive, the following interim data type mappings ar
330332

331333
To learn details about the properties, check [Lookup activity](control-flow-lookup-activity.md).
332334

333-
## Upgrade the Hive connector
334335

335-
Here are steps that help you upgrade the Hive connector:
336+
## <a name="differences-between-hive-version-20-and-version-10"></a> Hive connector lifecycle and upgrade
336337

337-
1. In **Edit linked service** page, select version 2.0 (Preview) and configure the linked service by referring to [Linked service properties version 2.0](#version-20).
338+
The following table shows the release stage and change logs for different versions of the Hive connector:
338339

339-
2. The data type mapping for the Hive linked service version 2.0 (Preview) is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Hive](#data-type-mapping-for-hive).
340+
| Version | Release stage | Change log |
341+
| :----------- | :------- | :------- |
342+
| Version 1.0 | End of support announced | / |
343+
| Version 2.0 | GA version available | • The self-hosted integration runtime version should be 5.54 or above. <br><br>• The default value of `enableSSL` is true. `enableServerCertificateValidation` is supported. <br>`trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported. <br><br>• TINYINT is read as SByte data type. <br><br>• TIMESTAMP is read as DateTimeOffset data type. <br><br>• Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is not supported. <br><br>• HiveServer1 and HiveThriftServer are not supported for `ServerType`. <br><br>• Username authentication type is not supported. SASL transport protocol only supports UsernameAndPassword authentication type. Binary transport protocol only supports Anonymous authentication type. <br><br>• `serviceDiscoveryMode`, `zooKeeperNameSpace` and `useNativeQuery` are not supported. |
340344

341-
## <a name="differences-between-hive-version-20-and-version-10"></a> Differences between Hive version 2.0 (Preview) and version 1.0
345+
### <a name="upgrade-the-hive-connector"></a> Upgrade the Hive connector from version 1.0 to version 2.0
342346

343-
The Hive connector version 2.0 (Preview) offers new functionalities and is compatible with most features of version 1.0. The following table shows the feature differences between version 2.0 (Preview) and version 1.0.
347+
1. In **Edit linked service** page, select version 2.0 and configure the linked service by referring to [Linked service properties version 2.0](#version-20).
344348

345-
| Version 2.0 (Preview) | Version 1.0 |
346-
|:--- |:--- |
347-
| Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is not supported.| Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is supported.|
348-
| HiveServer1 and HiveThriftServer are not supported for `ServerType`. | Support HiveServer1 and HiveThriftServer for `ServerType`. |
349-
| Username authentication type is not supported. <br><br>SASL transport protocol only supports UsernameAndPassword authentication type. Binary transport protocol only supports Anonymous authentication type. | Support Username authentication type. <br><br>SASL and Binary transport protocols support Anonymous, Username, UsernameAndPassword and WindowsAzureHDInsightService authentication types. |
350-
| `serviceDiscoveryMode`, `zooKeeperNameSpace` and `useNativeQuery` are not supported. | `serviceDiscoveryMode`, `zooKeeperNameSpace`, `useNativeQuery` are supported. |
351-
| The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.<br><br>`enableServerCertificateValidation` is supported.| The default value of `enableSSL` is false. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are supported.<br><br>`enableServerCertificateValidation` is not supported. |
352-
| The following mappings are used from Hive data types to interim service data type.<br><br>TINYINT -> SByte<br>TIMESTAMP -> DateTimeOffset | The following mappings are used from Hive data types to interim service data type.<br><br>TINYINT -> Int16 <br>TIMESTAMP -> DateTime |
349+
2. The data type mapping for the Hive linked service version 2.0 is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Hive](#data-type-mapping-for-hive).
353350

351+
3. Apply a self-hosted integration runtime with version 5.54 or above.
354352

355353
## Related content
356354
For a list of data stores supported as sources and sinks by the copy activity, see [supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).

0 commit comments

Comments
 (0)