You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
1. Create an HDInsight Spark **4.0** cluster with a storage account and a custom Azure virtual network. For information on creating a cluster in an Azure virtual network, see [Add HDInsight to an existing virtual network](../../hdinsight/hdinsight-plan-virtual-network-deployment.md#existingvnet).
63
+
1. Create an HDInsight Spark **5.1** cluster with a storage account and a custom Azure virtual network. For information on creating a cluster in an Azure virtual network, see [Add HDInsight to an existing virtual network](../../hdinsight/hdinsight-plan-virtual-network-deployment.md#existingvnet).
66
64
67
-
1. Create an HDInsight Interactive Query (LLAP) **4.0** cluster with the same storage account and Azure virtual network as the Spark cluster.
65
+
1. Create an HDInsight Interactive Query (LLAP) **5.1** cluster with the same storage account and Azure virtual network as the Spark cluster.
68
66
69
67
### Configure HWC settings
70
68
@@ -102,6 +100,20 @@ value. The value may be similar to: `thrift://iqgiro.rekufuk2y2cezcbowjkbwfnyvd.
102
100
103
101
1. Save changes and restart all affected components.
104
102
103
+
#### Additonal configurations for Spark and Hive
104
+
105
+
The following configuration needs to be done for **all** head and worker nodes of your Spark and Hive clusters.
106
+
107
+
1. Use [ssh command](../hdinsight-hadoop-linux-use-ssh-unix.md) to connect to your Apache Spark and Apache Hive nodes. Edit the command below by replacing CLUSTERNAME with the name of your cluster, and then enter the command:
1. Append the file content of /etc/hosts of hive cluster in /etc/hosts file of spark cluster and vice-versa.
114
+
115
+
1. Once all nodes are updated then, restart both the clusters.
116
+
105
117
### Configure HWC for Enterprise Security Package (ESP) clusters
106
118
107
119
The Enterprise Security Package (ESP) provides enterprise-grade capabilities like Active Directory-based authentication, multi-user support, and role-based access control for Apache Hadoop clusters in Azure HDInsight. For more information on ESP, see [Use Enterprise Security Package in HDInsight](../domain-joined/apache-domain-joined-architecture.md).
@@ -126,8 +138,19 @@ Apart from the configurations mentioned in the previous section, add the followi
126
138
127
139
* For instance, `hive/hn*.mjry42ikpruuxgs2qy2kpg4q5e.cx.internal.cloudapp.net@PKRSRVUQVMAE6J85.D2.INTERNAL.CLOUDAPP.NET`.
128
140
141
+
1. The following configuration needs to be done for **all** head and worker nodes of your Spark and Hive clusters.
142
+
143
+
* Use [ssh command](../hdinsight-hadoop-linux-use-ssh-unix.md) to connect to your Apache Spark and Apache Hive nodes. Edit the command below by replacing CLUSTERNAME with the name of your cluster, and then enter the command:
* Append tenant domain name (e.g. "abc.contoso.com”) in the last line of /etc/resolv.conf in head and worker nodes of your Spark and Hive clusters.
150
+
129
151
1. Save changes and restart components as needed.
130
152
153
+
131
154
## Hive Warehouse Connector usage
132
155
133
156
You can choose between a few different methods to connect to your Interactive Query cluster and execute queries using the Hive Warehouse Connector. Supported methods include the following tools:
0 commit comments