You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-plan-virtual-network-deployment.md
+20-17Lines changed: 20 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,20 +5,20 @@ author: hrasheed-msft
5
5
ms.author: hrasheed
6
6
ms.reviewer: jasonh
7
7
ms.service: hdinsight
8
-
ms.custom: hdinsightactive
9
8
ms.topic: conceptual
10
-
ms.date: 07/23/2019
9
+
ms.custom: hdinsightactive
10
+
ms.date: 02/25/2020
11
11
---
12
12
13
13
# Plan a virtual network for Azure HDInsight
14
14
15
-
This article provides background information on using [Azure Virtual Networks](../virtual-network/virtual-networks-overview.md) with Azure HDInsight. It also discusses design and implementation decisions that must be made before you can implement a virtual network for your HDInsight cluster. Once the planning phase is finished, you can proceed to [Create virtual networks for Azure HDInsight clusters](hdinsight-create-virtual-network.md). For more information on HDInsight management IP addresses that are needed to properly configure network security groups and user-defined routes, see [HDInsight management IP addresses](hdinsight-management-ip-addresses.md).
15
+
This article provides background information on using [Azure Virtual Networks](../virtual-network/virtual-networks-overview.md)(VNets) with Azure HDInsight. It also discusses design and implementation decisions that must be made before you can implement a virtual network for your HDInsight cluster. Once the planning phase is finished, you can proceed to [Create virtual networks for Azure HDInsight clusters](hdinsight-create-virtual-network.md). For more information on HDInsight management IP addresses that are needed to properly configure network security groups (NSGs) and user-defined routes, see [HDInsight management IP addresses](hdinsight-management-ip-addresses.md).
16
16
17
17
Using an Azure Virtual Network enables the following scenarios:
18
18
19
19
* Connecting to HDInsight directly from an on-premises network.
20
20
* Connecting HDInsight to data stores in an Azure Virtual network.
21
-
* Directly accessing [Apache Hadoop](https://hadoop.apache.org/) services that are not available publicly over the internet. For example, [Apache Kafka](https://kafka.apache.org/) APIs or the [Apache HBase](https://hbase.apache.org/) Java API.
21
+
* Directly accessing Apache Hadoop services that aren't available publicly over the internet. For example, Apache Kafka APIs or the Apache HBase Java API.
22
22
23
23
> [!IMPORTANT]
24
24
> Creating an HDInsight cluster in a VNET will create several networking resources, such as NICs and load balancers. Do **not** delete these networking resources, as they are needed for your cluster to function correctly with the VNET.
@@ -31,7 +31,7 @@ The following are the questions that you must answer when planning to install HD
31
31
32
32
* Do you need to install HDInsight into an existing virtual network? Or are you creating a new network?
33
33
34
-
If you are using an existing virtual network, you may need to modify the network configuration before you can install HDInsight. For more information, see the [add HDInsight to an existing virtual network](#existingvnet) section.
34
+
If you're using an existing virtual network, you may need to modify the network configuration before you can install HDInsight. For more information, see the [add HDInsight to an existing virtual network](#existingvnet) section.
35
35
36
36
* Do you want to connect the virtual network containing HDInsight to another virtual network or your on-premises network?
37
37
@@ -59,19 +59,19 @@ Use the steps in this section to discover how to add a new HDInsight to an exist
59
59
2. Do you use network security groups, user-defined routes, or Virtual Network Appliances to restrict traffic into or out of the virtual network?
60
60
61
61
As a managed service, HDInsight requires unrestricted access to several IP addresses in the Azure data center. To allow communication with these IP addresses, update any existing network security groups or user-defined routes.
62
-
63
-
HDInsight hosts multiple services, which use a variety of ports. Do not block traffic to these ports. For a list of ports to allow through virtual appliance firewalls, see the Security section.
64
-
62
+
63
+
HDInsight hosts multiple services, which use a variety of ports. Don't block traffic to these ports. For a list of ports to allow through virtual appliance firewalls, see the Security section.
64
+
65
65
To find your existing security configuration, use the following Azure PowerShell or Azure CLI commands:
66
66
67
67
* Network security groups
68
68
69
69
Replace `RESOURCEGROUP` with the name of the resource group that contains the virtual network, and then enter the command:
az network nsg list --resource-group RESOURCEGROUP
77
77
```
@@ -120,7 +120,7 @@ Azure provides name resolution for Azure services that are installed in a virtua
120
120
121
121
Both these nodes can communicate directly with each other, and other nodes in HDInsight, by using internal DNS names.
122
122
123
-
The default name resolution does __not__ allow HDInsight to resolve the names of resources in networks that are joined to the virtual network. For example, it is common to join your on-premises network to the virtual network. With only the default name resolution, HDInsight cannot access resources in the on-premises network by name. The opposite is also true, resources in your on-premises network cannot access resources in the virtual network by name.
123
+
The default name resolution does __not__ allow HDInsight to resolve the names of resources in networks that are joined to the virtual network. For example, it's common to join your on-premises network to the virtual network. With only the default name resolution, HDInsight can't access resources in the on-premises network by name. The opposite is also true, resources in your on-premises network can't access resources in the virtual network by name.
124
124
125
125
> [!WARNING]
126
126
> You must create the custom DNS server and configure the virtual network to use it before creating the HDInsight cluster.
@@ -136,7 +136,7 @@ To enable name resolution between the virtual network and resources in joined ne
136
136
4. Configure forwarding between the DNS servers. The configuration depends on the type of remote network.
137
137
138
138
* If the remote network is an on-premises network, configure DNS as follows:
139
-
139
+
140
140
* __Custom DNS__ (in the virtual network):
141
141
142
142
* Forward requests for the DNS suffix of the virtual network to the Azure recursive resolver (168.63.129.16). Azure handles requests for resources in the virtual network
@@ -230,12 +230,12 @@ For more information on controlling outbound traffic from HDInsight clusters, se
230
230
231
231
#### Forced tunneling to on-premises
232
232
233
-
Forced tunneling is a user-defined routing configuration where all traffic from a subnet is forced to a specific network or location, such as your on-premises network. HDInsight does __not__ support forced tunneling of traffic to on-premises networks.
233
+
Forced tunneling is a user-defined routing configuration where all traffic from a subnet is forced to a specific network or location, such as your on-premises network. HDInsight does __not__ support forced tunneling of traffic to on-premises networks.
234
234
235
235
## <a id="hdinsight-ip"></a> Required IP addresses
236
236
237
-
If you use network security groups or user-defined routes to control traffic, please see [HDInsight management IP addresses](hdinsight-management-ip-addresses.md).
238
-
237
+
If you use network security groups or user-defined routes to control traffic, see [HDInsight management IP addresses](hdinsight-management-ip-addresses.md).
238
+
239
239
## <a id="hdinsight-ports"></a> Required ports
240
240
241
241
If you plan on using a **firewall** and access the cluster from outside on certain ports, you might need to allow traffic on those ports needed for your scenario. By default, no special whitelisting of ports is needed as long as the azure management traffic explained in the previous section is allowed to reach cluster on port 443.
@@ -246,13 +246,16 @@ For more information on firewall rules for virtual appliances, see the [virtual
246
246
247
247
## Load balancing
248
248
249
-
When you create an HDInsight cluster, a load balancer is created as well. The type of this load balancer is at the [basic SKU level](../load-balancer/concepts-limitations.md#skus) which has certain constraints. One of these constraints is that if you have two virtual networks in different regions, you cannot connect to basic load balancers. See [virtual networks FAQ: constraints on global vnet peering](../virtual-network/virtual-networks-faq.md#what-are-the-constraints-related-to-global-vnet-peering-and-load-balancers), for more information.
249
+
When you create an HDInsight cluster, a load balancer is created as well. The type of this load balancer is at the [basic SKU level](../load-balancer/concepts-limitations.md#skus), which has certain constraints. One of these constraints is that if you have two virtual networks in different regions, you cannot connect to basic load balancers. See [virtual networks FAQ: constraints on global vnet peering](../virtual-network/virtual-networks-faq.md#what-are-the-constraints-related-to-global-vnet-peering-and-load-balancers), for more information.
250
250
251
251
## Transport Layer Security
252
252
253
253
Connections to the cluster via the public cluster endpoint `https://<clustername>.azurehdinsight.net` are proxied through cluster gateway nodes. These connections are secured using a protocol called TLS. Enforcing higher versions of TLS on gateways improves the security for these connections. For more information on why you should use newer versions of TLS, see [Solving the TLS 1.0 Problem](https://docs.microsoft.com/security/solving-tls1-problem).
254
254
255
-
You can control the minimum TLS version(s) supported on the gateway nodes for your HDInsight cluster by using the *minSupportedTlsVersion* property in a resource manager template at deployment time. For a sample template, see [HDInsight minimum TLS 1.2 Quickstart template](https://github.com/Azure/azure-quickstart-templates/tree/master/101-hdinsight-minimum-tls). This property supports three values: “1.0”, “1.1” and “1.2”, which correspond to TLS 1.0+, TLS 1.1+ and TLS 1.2+ respectively. By default, without specifying this property, Azure HDInsight clusters accept TLS 1.2 connections on public HTTPS endpoints, as well as older versions for backward compatibility. Eventually, HDInsight will enforce TLS 1.2 or later on all gateway node connections.
255
+
By default, Azure HDInsight clusters accept TLS 1.2 connections on public HTTPS endpoints, as well as older versions for backward compatibility. You can control the minimum TLS version supported on the gateway nodes during cluster creation using either the Azure portal, or a resource manager template. For the portal, select the TLS version from the **Security + networking** tab during cluster creation. For a resource manager template at deployment time, use the **minSupportedTlsVersion** property. For a sample template, see [HDInsight minimum TLS 1.2 Quickstart template](https://github.com/Azure/azure-quickstart-templates/tree/master/101-hdinsight-minimum-tls). This property supports three values: "1.0", "1.1" and "1.2", which correspond to TLS 1.0+, TLS 1.1+ and TLS 1.2+ respectively.
256
+
257
+
> [!IMPORTANT]
258
+
> Starting on June 30, 2020, Azure HDInsight will enforce TLS 1.2 or later versions for all HTTPS connections. We recommend that you ensure that all your clients are ready to handle TLS 1.2 or later versions. For more information, see [Azure HDInsight TLS 1.2 Enforcement](https://azure.microsoft.com/updates/azure-hdinsight-tls-12-enforcement/).
0 commit comments