Skip to content

Commit 85fc3a4

Browse files
Lucky-Wang16Lucky-Wang16
committed
Update Hive v2 doc
1 parent 722027a commit 85fc3a4

File tree

1 file changed

+106
-2
lines changed

1 file changed

+106
-2
lines changed

‎articles/data-factory/connector-hive.md

Lines changed: 106 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ author: jianleishen
66
ms.subservice: data-movement
77
ms.custom: synapse
88
ms.topic: conceptual
9-
ms.date: 09/12/2024
9+
ms.date: 05/27/2025
1010
ms.author: jianleishen
1111
---
1212

@@ -71,7 +71,66 @@ The following sections provide details about properties that are used to define
7171

7272
## Linked service properties
7373

74-
The following properties are supported for Hive linked service:
74+
The Hive connector now supports version 2.0 (Preview). Refer to this [section](#upgrade-the-hive-connector) to upgrade your Hive connector version from version 1.0. For the property details, see the corresponding sections.
75+
76+
- [Version 2.0 (Preview)](#version-20)
77+
- [Version 1.0](#version-10)
78+
79+
### <a name="version-20"></a> Version 2.0 (Preview)
80+
81+
The Hive linked service supports the following properties when apply version 2.0 (Preview):
82+
83+
| Property | Description | Required |
84+
|:--- |:--- |:--- |
85+
| type | The type property must be set to: **Hive** | Yes |
86+
| version | The version that you specify. The value is `2.0`. | Yes |
87+
| host | IP address or host name of the Hive server. | Yes |
88+
| port | The TCP port that the Hive server uses to listen for client connections. If you connect to Azure HDInsight, specify port as 443. | Yes |
89+
| serverType | The type of Hive server. <br/>Allowed value is: **HiveServer2** | No |
90+
| thriftTransportProtocol | The transport protocol to use in the Thrift layer. <br/>Allowed value is: **HTTP** | No |
91+
| authenticationType | The authentication method used to access the Hive server. <br/>Allowed values are: **Anonymous**, **UsernameAndPassword**, **WindowsAzureHDInsightService**. Kerberos authentication is not supported now. | Yes |
92+
| username | The user name that you use to access Hive Server. | No |
93+
| password | The password corresponding to the user. Mark this field as a SecureString to store it securely, or [reference a secret stored in Azure Key Vault](store-credentials-in-key-vault.md). | No |
94+
| httpPath | The partial URL corresponding to the Hive server. | No |
95+
| enableSsl | Specifies whether the connections to the server are encrypted using TLS. The default value is true. | No |
96+
| enableServerCertificateValidation | Specify whether to enable server SSL certificate validation when you connect.
97+
Always use System Trust Store. The default value is true. | No |
98+
| storageReference | A reference to the linked service of the storage account used for staging data in mapping data flow. This is required only when using the Hive linked service in mapping data flow. | No |
99+
| connectVia | The [Integration Runtime](concepts-integration-runtime.md) to be used to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, it uses the default Azure Integration Runtime. |No |
100+
101+
**Example:**
102+
103+
```json
104+
{
105+
"name": "HiveLinkedService",
106+
"properties": {
107+
"type": "Hive",
108+
"version": "2.0",
109+
"typeProperties": {
110+
"host" : "<host>",
111+
"port" : "<port>",
112+
"authenticationType" : "WindowsAzureHDInsightService",
113+
"username" : "<username>",
114+
"password": {
115+
"type": "SecureString",
116+
"value": "<password>"
117+
},
118+
"serverType": "HiveServer2",
119+
"thriftTransportProtocol": "HTTP",
120+
"enableSsl": true,
121+
"enableServerCertificateValidation": true
122+
},
123+
"connectVia": {
124+
"referenceName": "<name of Integration Runtime>",
125+
"type": "IntegrationRuntimeReference"
126+
}
127+
}
128+
}
129+
```
130+
131+
### Version 1.0
132+
133+
The following properties are supported for Hive linked service when apply version 1.0:
75134

76135
| Property | Description | Required |
77136
|:--- |:--- |:--- |
@@ -241,10 +300,55 @@ source(
241300
a. Check the setting "hive.resultset.use.unique.column.names" in Hive server side and set it to false.
242301
b. Use column mapping to rename the column name.
243302

303+
## Data type mapping for Hive
304+
305+
When you copy data from and to Hive, the following interim data type mappings are used within the service. To learn about how the copy activity maps the source schema and data type to the sink, see [Schema and data type mappings](copy-activity-schema-and-type-mapping.md).
306+
307+
| Hive data type | Interim service data type (for version 2.0 (Preview)) | Interim service data type (for version 1.0) |
308+
|:--- |:--- |:--- |
309+
| TINYINT | Sbyte | Int16 |
310+
| SMALLINT | Int16 | Int16 |
311+
| INT | Int32 | Int32 |
312+
| BIGINT | Int32 | Int64 |
313+
| BOOLEAN |Boolean | Boolean |
314+
| FLOAT | Single | Single |
315+
| DOUBLE | Double | Double |
316+
| DECIMAL | Decimal | Decimal |
317+
| STRING | String | String |
318+
| VARCHAR | String | String |
319+
| CHAR | String | String |
320+
| TIMESTAMP |DateTimeOffset | DateTime |
321+
| DATE | DateTime | DateTime |
322+
| BINARY | Byte[] | Byte[] |
323+
| ARRAY | String | String |
324+
| MAP | String | String |
325+
| STRUCT | String | String |
326+
244327
## Lookup activity properties
245328

246329
To learn details about the properties, check [Lookup activity](control-flow-lookup-activity.md).
247330

331+
## Upgrade the Hive connector
332+
333+
Here are steps that help you upgrade the Hive connector:
334+
335+
1. In **Edit linked service** page, select version 2.0 (Preview) and configure the linked service by referring to [Linked service properties version 2.0](#version-20).
336+
337+
2. The data type mapping for the Hive linked service version 2.0 (Preview) is different from that for the version 1.0. To learn the latest data type mapping, see [Data type mapping for Hive](#data-type-mapping-for-hive).
338+
339+
## Differences between Hive version 2.0 (Preview) and version 1.0
340+
341+
The Hive connector version 2.0 (Preview) offers new functionalities and is compatible with most features of version 1.0. The following table shows the feature differences between version 2.0 (Preview) and version 1.0.
342+
343+
| Version 2.0 (Preview) | Version 1.0 |
344+
|:--- |:--- |
345+
| Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is not supported.| Using ';' to separate multiple hosts (only when serviceDiscoveryMode is enabled) is supported.|
346+
| `ServerType` does not support `HiveServer1` and `HiveThriftServer` | `ServerType` supports `HiveServer1` and `HiveThriftServer`|
347+
| `authenticationType` does not support `Username`. | `authenticationType` supports `Username`. |
348+
| The default value of `enableSSL` is true. <br><br>`enableServerCertificateValidation` is supported.<br><br>`serviceDiscoveryMode`, `zooKeeperNameSpace`, `useNativeQuery`, `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are not supported.| The default value of `enableSSL` is false.<br><br>`enableServerCertificateValidation` is not supported.<br><br>`serviceDiscoveryMode`, `zooKeeperNameSpace`, `useNativeQuery`, `trustedCertPath`, `useSystemTrustStore`, `allowHostNameCNMismatch` and `allowSelfSignedServerCert` are supported. |
349+
| `thriftTransportProtocol` does not support `Binary` and `SASL`.| `thriftTransportProtocol` supports `Binary` and `SASL`. |
350+
| The following mappings are used from Hive data types to interim service data type.<br><br>TINYINT -> SByte<br>TIMESTAMP -> DateTimeOffset | The following mappings are used from Hive data types to interim service data type.<br><br>TINYINT -> Int16 <br>TIMESTAMP -> DateTime |
351+
248352

249353
## Related content
250354
For a list of data stores supported as sources and sinks by the copy activity, see [supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).

0 commit comments

Comments
 (0)