|
1 | 1 | ---
|
2 | 2 | title: Apache Spark & Hive - Hive Warehouse Connector - Azure HDInsight
|
3 | 3 | description: Learn how to integrate Apache Spark and Apache Hive with the Hive Warehouse Connector on Azure HDInsight.
|
4 |
| -author: nakhanha |
5 |
| -ms.author: nakhanha |
| 4 | +author: hrasheed-msft |
| 5 | +ms.author: hrasheed |
6 | 6 | ms.reviewer: hrasheed
|
7 | 7 | ms.service: hdinsight
|
8 | 8 | ms.topic: conceptual
|
9 |
| -ms.date: 10/08/2019 |
| 9 | +ms.date: 03/02/2020 |
10 | 10 | ---
|
11 | 11 |
|
12 | 12 | # Integrate Apache Spark and Apache Hive with the Hive Warehouse Connector
|
@@ -49,17 +49,17 @@ Copy the node information from the `/etc/hosts` file on headnode0 of your Intera
|
49 | 49 |
|
50 | 50 | #### From your Interactive Query cluster
|
51 | 51 |
|
52 |
| -1. Navigate to the cluster's Apache Ambari home page using `https://LLAPCLUSTERNAME.azurehdinsight.net` where `LLAPCLUSTERNAME` is the name of your Interactive Query cluster. |
| 52 | +1. Navigate to the cluster's Apache Ambari Hive page using `https://LLAPCLUSTERNAME.azurehdinsight.net/#/main/services/HIVE/configs` where `LLAPCLUSTERNAME` is the name of your Interactive Query cluster. |
53 | 53 |
|
54 |
| -1. Navigate to **Hive** > **CONFIGS** > **Advanced** > **Advanced hive-site** > **hive.zookeeper.quorum** and note the value. The value may be similar to: `zk0-iqgiro.rekufuk2y2cezcbowjkbwfnyvd.bx.internal.cloudapp.net:2181,zk1-iqgiro.rekufuk2y2cezcbowjkbwfnyvd.bx.internal.cloudapp.net:2181,zk4-iqgiro.rekufuk2y2cezcbowjkbwfnyvd.bx.internal.cloudapp.net:2181`. |
| 54 | +1. Navigate to **Advanced** > **General** > **hive.metastore.uris** and note the value. The value may be similar to: `thrift://iqgiro.rekufuk2y2cezcbowjkbwfnyvd.bx.internal.cloudapp.net:9083,thrift://hn1-iqgiro.rekufuk2y2cezcbowjkbwfnyvd.bx.internal.cloudapp.net:9083`. |
55 | 55 |
|
56 |
| -1. Navigate to **Hive** > **CONFIGS** > **Advanced** > **General** > **hive.metastore.uris** and note the value. The value may be similar to: `thrift://iqgiro.rekufuk2y2cezcbowjkbwfnyvd.bx.internal.cloudapp.net:9083,thrift://hn1-iqgiro.rekufuk2y2cezcbowjkbwfnyvd.bx.internal.cloudapp.net:9083`. |
| 56 | +1. Navigate to **Advanced** > **Advanced hive-site** > **hive.zookeeper.quorum** and note the value. The value may be similar to: `zk0-iqgiro.rekufuk2y2cezcbowjkbwfnyvd.bx.internal.cloudapp.net:2181,zk1-iqgiro.rekufuk2y2cezcbowjkbwfnyvd.bx.internal.cloudapp.net:2181,zk4-iqgiro.rekufuk2y2cezcbowjkbwfnyvd.bx.internal.cloudapp.net:2181`. |
57 | 57 |
|
58 | 58 | #### From your Apache Spark cluster
|
59 | 59 |
|
60 |
| -1. Navigate to the cluster's Apache Ambari home page using `https://SPARKCLUSTERNAME.azurehdinsight.net` where `SPARKCLUSTERNAME` is the name of your Apache Spark cluster. |
| 60 | +1. Navigate to the cluster's Apache Ambari Hive page using `https://SPARKCLUSTERNAME.azurehdinsight.net/#/main/services/HIVE/configs` where `SPARKCLUSTERNAME` is the name of your Apache Spark cluster. |
61 | 61 |
|
62 |
| -1. Navigate to **Hive** > **CONFIGS** > **Advanced** > **Advanced hive-interactive-site** > **hive.llap.daemon.service.hosts** and note the value. The value may be similar to: `@llap0`. |
| 62 | +1. Navigate to **Advanced** > **Advanced hive-interactive-site** > **hive.llap.daemon.service.hosts** and note the value. The value may be similar to: `@llap0`. |
63 | 63 |
|
64 | 64 | ### Configure Spark cluster settings
|
65 | 65 |
|
@@ -169,7 +169,7 @@ Spark doesn’t natively support writing to Hive’s managed ACID tables. Using
|
169 | 169 | ```scala
|
170 | 170 | hive.table("sampletable_colorado").show()
|
171 | 171 | ```
|
172 |
| - |
| 172 | + |
173 | 173 | 
|
174 | 174 |
|
175 | 175 | ### Structured streaming writes
|
@@ -256,5 +256,5 @@ Use **Ctrl + C** to stop netcat on the second SSH session. Use `:q` to exit spar
|
256 | 256 |
|
257 | 257 | ## Next steps
|
258 | 258 |
|
259 |
| -* [Use Interactive Query with HDInsight](https://docs.microsoft.com/azure/hdinsight/interactive-query/apache-interactive-query-get-started) |
| 259 | +* [Use Interactive Query with HDInsight](./apache-interactive-query-get-started.md) |
260 | 260 | * [Examples of interacting with Hive Warehouse Connector using Zeppelin, Livy, spark-submit, and pyspark](https://community.hortonworks.com/articles/223626/integrating-apache-hive-with-apache-spark-hive-war.html)
|
0 commit comments