You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/interactive-query/apache-hive-warehouse-connector.md
+9-9Lines changed: 9 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,8 +20,8 @@ Apache Hive offers support for database transactions that are Atomic, Consistent
20
20
Apache Spark has a Structured Streaming API that gives streaming capabilities not available in Apache Hive. Beginning with HDInsight 4.0, Apache Spark 2.3.1 & above, and Apache Hive 3.1.0 have separate metastore catalogs, which make interoperability difficult.
21
21
22
22
The Hive Warehouse Connector (HWC) makes it easier to use Spark and Hive together. The HWC library loads data from LLAP daemons to Spark executors in parallel. This process makes it more efficient and adaptable than a standard JDBC connection from Spark to Hive. This brings out two different execution modes for HWC:
23
-
>- Hive JDBC mode via HiveServer2
24
-
>- Hive LLAP mode using LLAP daemons **[Recommended]**
23
+
- Hive JDBC mode via HiveServer2
24
+
- Hive LLAP mode using LLAP daemons **[Recommended]**
25
25
26
26
By default, HWC is configured to use Hive LLAP daemons.
27
27
For executing Hive queries (both read and write) using the above modes with their respective APIs, see [HWC APIs](./hive-warehouse-connector-apis.md).
@@ -99,7 +99,7 @@ value. The value may be similar to: `thrift://iqgiro.rekufuk2y2cezcbowjkbwfnyvd.
99
99
100
100
1. Save changes and restart all affected components.
101
101
102
-
#### Additonal configurations for Spark and Hive
102
+
#### Additional configurations for Spark and Hive
103
103
104
104
The following configuration needs to be done for **all** head and worker nodes of your Spark and Hive clusters.
105
105
@@ -129,7 +129,7 @@ Apart from the configurations mentioned in the previous section, add the followi
129
129
130
130
* From a web browser, navigate to `https://CLUSTERNAME.azurehdinsight.net/#/main/services/HIVE/summary` where CLUSTERNAME is the name of your Interactive Query cluster. Click on **HiveServer2 Interactive**. You'll see the Fully Qualified Domain Name (FQDN) of the head node on which LLAP is running as shown in the screenshot. Replace `<llap-headnode>` with this value.
131
131
132
-
:::image type="content" source="./media/apache-hive-warehouse-connector/head-node-hive-server-interactive.png" alt-text="hive warehouse connector Head Node." border="true":::
132
+
:::image type="content" source="./media/apache-hive-warehouse-connector/head-node-hive-server-interactive.png" alt-text="Screenshot of hive warehouse connector Head Node." border="true":::
133
133
134
134
* Use [ssh command](../hdinsight-hadoop-linux-use-ssh-unix.md) to connect to your Interactive Query cluster. Look for `default_realm` parameter in the `/etc/krb5.conf` file. Replace `<AAD-DOMAIN>` with this value as an uppercase string, otherwise the credential won't be found.
135
135
@@ -256,21 +256,21 @@ kinit USERNAME
256
256
hive.executeQuery("SELECT * FROM demo").show()
257
257
```
258
258
259
-
:::image type="content" source="./media/apache-hive-warehouse-connector/hive-warehouse-connector-table-before-ranger-policy.png" alt-text="demo table before applying ranger policy." border="true":::
259
+
:::image type="content" source="./media/apache-hive-warehouse-connector/hive-warehouse-connector-table-before-ranger-policy.png" alt-text="Screenshot of demo table before applying ranger policy." border="true":::
260
260
261
261
1. Apply a column masking policy that only shows the last four characters of the column.
262
262
1. Go to the RangerAdminUI at `https://LLAPCLUSTERNAME.azurehdinsight.net/ranger/`.
263
263
1. Click on the Hive service for your cluster under **Hive**.
264
-
:::image type="content" source="./media/apache-hive-warehouse-connector/hive-warehouse-connector-ranger-service-manager.png" alt-text="ranger service manager." border="true":::
264
+
:::image type="content" source="./media/apache-hive-warehouse-connector/hive-warehouse-connector-ranger-service-manager.png" alt-text="Screenshot of ranger service manager." border="true":::
265
265
1. Click on the **Masking** tab and then**AddNewPolicy**
1. Provide a desired policy name. Selectdatabase: **Default**, Hivetable: **demo**, Hivecolumn: **name**, User:**rsadmin2**, AccessTypes:**select**, and **Partialmask: show last 4** from the **SelectMaskingOption** menu. Click**Add**.
0 commit comments