Merge pull request #300397 from Lucky-Wang16/0526-Update_Hive_v2

v-ccolin · web-flow · commit b2d687894d01 · 2025-06-06T11:06:26.000+01:00
Update Hive v2 doc
diff --git a/articles/data-factory/connector-hive.md b/articles/data-factory/connector-hive.md
@@ -6,7 +6,7 @@ author: jianleishen
 ms.subservice: data-movement
 ms.custom: synapse
 ms.topic: conceptual
-ms.date: 09/12/2024
+ms.date: 05/27/2025
 ms.author: jianleishen
 ---
 
@@ -15,6 +15,9 @@ ms.author: jianleishen
 
 This article outlines how to use the Copy Activity in an Azure Data Factory or Synapse Analytics pipeline to copy data from Hive. It builds on the [copy activity overview](copy-activity-overview.md) article that presents a general overview of copy activity.
 
+> [!IMPORTANT]
+> The Hive connector version 2.0 (Preview) provides improved native Hive support. If you are using the Hive connector version 1.0 in your solution, please [upgrade your Hive connector](#upgrade-the-hive-connector) before **September 30, 2025**. Refer to this [section](#differences-between-hive-version-20-and-version-10) for details on the difference between version 2.0 (Preview) and version 1.0.
+
 ## Supported capabilities
 
 This Hive connector is supported for the following capabilities:
@@ -71,7 +74,65 @@ The following sections provide details about properties that are used to define
 
 ## Linked service properties
 
-The following properties are supported for Hive linked service:
+The Hive connector now supports version 2.0 (Preview). Refer to this [section](#upgrade-the-hive-connector) to upgrade your Hive connector version from version 1.0. For the property details, see the corresponding sections.
+
+- [Version 2.0 (Preview)](#version-20)
+- [Version 1.0](#version-10)
+
+### <a name="version-20"></a> Version 2.0 (Preview)
+
+The Hive linked service supports the following properties when apply version 2.0 (Preview):
+
+| Property | Description | Required |
+|:--- |:--- |:--- |
+| type | The type property must be set to: **Hive** | Yes |
+| version | The version that you specify. The value is `2.0`. | Yes |
+| host | IP address or host name of the Hive server.  | Yes |
+| port | The TCP port that the Hive server uses to listen for client connections. If you connect to Azure HDInsight, specify port as 443. | Yes |
+| serverType | The type of Hive server. <br/>Allowed value is: **HiveServer2** | No |
+| thriftTransportProtocol | The transport protocol to use in the Thrift layer. <br/>Allowed value are: **Binary**, **SASL**, **HTTP** | No |
+| authenticationType | The authentication method used to access the Hive server. <br/>Allowed values are: **Anonymous**, **UsernameAndPassword**, **WindowsAzureHDInsightService**. Kerberos authentication is not supported now. | Yes |
+| username | The user name that you use to access Hive Server.  | No |
+| password | The password corresponding to the user. Mark this field as a SecureString to store it securely, or [reference a secret stored in Azure Key Vault](store-credentials-in-key-vault.md). | No |
+| httpPath | The partial URL corresponding to the Hive server.  | No |
+| enableSsl | Specifies whether the connections to the server are encrypted using TLS. The default value is true.  | No |
+| enableServerCertificateValidation | Specify whether to enable server SSL certificate validation when you connect. Always use System Trust Store. The default value is true. | No |
+| storageReference | A reference to the linked service of the storage account used for staging data in mapping data flow. This is required only when using the Hive linked service in mapping data flow. | No |
+| connectVia | The [Integration Runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. |No |
+
+**Example:**
+
+```json
+{
+    "name": "HiveLinkedService",
+    "properties": {
+        "type": "Hive",
+        "version": "2.0",
+        "typeProperties": {
+            "host" : "<host>",
+            "port" : "<port>",
+            "authenticationType" : "WindowsAzureHDInsightService",
+            "username" : "<username>",
+            "password": {
+                "type": "SecureString",
+                "value": "<password>"
+            },
+            "serverType": "HiveServer2",
+            "thriftTransportProtocol": "HTTP",
+            "enableSsl": true,
+            "enableServerCertificateValidation": true
+        },
+        "connectVia": {
+            "referenceName": "<name of Integration Runtime>",
+            "type": "IntegrationRuntimeReference"
+        }
+    }
+}
+```
+
+### Version 1.0
+
+The following properties are supported for Hive linked service when apply version 1.0:
 
 | Property | Description | Required |
 |:--- |:--- |:--- |
@@ -241,10 +302,55 @@ source(
   a. Check the setting "hive.resultset.use.unique.column.names" in Hive server side and set it to false.
   b. Use column mapping to rename the column name.
 
+## Data type mapping for Hive
+
+When you copy data from and to Hive, the following interim data type mappings are used within the service. To learn about how the copy activity maps the source schema and data type to the sink, see [Schema and data type mappings](copy-activity-schema-and-type-mapping.md).
+
+| Hive data type | Interim service data type (for version 2.0 (Preview)) | Interim service data type (for version 1.0) |
+|:--- |:--- |:--- |
+| TINYINT | Sbyte | Int16 |
+| SMALLINT | Int16 | Int16 |
+| INT | Int32 | Int32 |
+| BIGINT | Int32 | Int64 |
+| BOOLEAN |Boolean | Boolean |
+| FLOAT | Single | Single |
+| DOUBLE  | Double | Double |
+| DECIMAL | Decimal | Decimal |
+| STRING | String | String |
+| VARCHAR | String | String |
+| CHAR | String | String |
+| TIMESTAMP |DateTimeOffset | DateTime |
+| DATE | DateTime | DateTime |
+| BINARY | Byte[] | Byte[] |
+| ARRAY |  String | String |
+| MAP |  String | String |
+| STRUCT | String | String |
+
 ## Lookup activity properties
 
 To learn details about the properties, check [Lookup activity](control-flow-lookup-activity.md).
 
+## Upgrade the Hive connector 
+
+Here are steps that help you upgrade the Hive connector: 
+
+1. In **Edit linked service** page, select version 2.0 (Preview) and configure the linked service by referring to [Linked service properties version 2.0](#version-20). 
+
+2. The data type mapping for the Hive linked service version 2.0 (Preview) is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Hive](#data-type-mapping-for-hive).
+
+## <a name="differences-between-hive-version-20-and-version-10"></a> Differences between Hive version 2.0 (Preview) and version 1.0 
+
+The Hive connector version 2.0 (Preview) offers new functionalities and is compatible with most features of version 1.0. The following table shows the feature differences between version 2.0 (Preview) and version 1.0. 
+
+| Version 2.0 (Preview) | Version 1.0 |
+|:--- |:--- |
+| Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is not supported.| Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is supported.|
+| HiveServer1 and HiveThriftServer are not supported for `ServerType`. | Support HiveServer1 and HiveThriftServer for `ServerType`. |
+| Username authentication type is not supported. <br><br>SASL transport protocol only supports UsernameAndPassword authentication type. Binary transport protocol only supports Anonymous authentication type. | Support Username authentication type. <br><br>SASL and Binary transport protocols support Anonymous, Username, UsernameAndPassword and WindowsAzureHDInsightService authentication types. |
+| `serviceDiscoveryMode`, `zooKeeperNameSpace` and `useNativeQuery` are not supported. | `serviceDiscoveryMode`, `zooKeeperNameSpace`, `useNativeQuery` are supported. |
+| The default value of `enableSSL` is true. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.<br><br>`enableServerCertificateValidation` is supported.| The default value of `enableSSL` is false. `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are supported.<br><br>`enableServerCertificateValidation` is not supported.  |
+| The following mappings are used from Hive data types to interim service data type.<br><br>TINYINT -> SByte<br>TIMESTAMP -> DateTimeOffset | The following mappings are used from Hive data types to interim service data type.<br><br>TINYINT -> Int16 <br>TIMESTAMP -> DateTime |  
+
 
 ## Related content
 For a list of data stores supported as sources and sinks by the copy activity, see [supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).
diff --git a/articles/data-factory/connector-release-stages-and-timelines.md b/articles/data-factory/connector-release-stages-and-timelines.md
@@ -38,6 +38,8 @@ For comprehensive details on support levels and recommended usage at each stage,
 | [Greenplum](connector-greenplum.md)                        | version 2.0    | GA                   | /                   | /                   |
 |                                    | version 1.0    | End of support announced                                | August 31, 2025     | September 30, 2025                    |
 | [HBase](connector-hbase.md)                              | /              | End of support                    | December 31, 2024     | December 31, 2024     |
+| [Hive](connector-hive.md)                          | version 2.0    | GA                                | /                   | /                   |
+|                                    | version 1.0    | End of support announced          | September 30, 2025         | October 31, 2025    |
 | [Magento (Preview)](connector-magento.md)                  | /              | End of support                    | December 31, 2024     | December 31, 2024     |
 | [MariaDB](connector-mariadb.md)                            | version 2.0    | GA                                | /                   | /                   |
 |                                    | version 1.0    | End of support                    | October 31, 2024      | September 30, 2025    |
diff --git a/articles/data-factory/media/connector-hive/configure-hive-linked-service.png b/articles/data-factory/media/connector-hive/configure-hive-linked-service.png
