Skip to content

Commit 2db7652

Browse files
Update load-balancer-migration-guidelines.md
1 parent fc7073e commit 2db7652

File tree

1 file changed

+72
-69
lines changed

1 file changed

+72
-69
lines changed
Lines changed: 72 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -1,118 +1,121 @@
11
---
2-
title: Load balancer migration guidelines for Azure HDInsight
3-
description: Guidelines to perform load balancer migration for Azure HDInsight.
2+
title: Basic Load balancer deprecation - Guidelines for Azure HDInsight
3+
description: Guidelines to transition to standard load balancer for Azure HDInsight.
44
ms.service: azure-hdinsight
55
ms.topic: how-to
66
ms.date: 09/25/2024
77
---
88

9-
# Load balancer migration guidelines
9+
# Basic Load balancer deprecation: Guidelines for Azure HDInsight
1010

1111
This article describes the details about the impact on HDInsight clusters and the necessary steps required as HDInsight service is transitioning to use standard load balancers for all its cluster configurations.
1212

1313
This transition is done in line with the announcement of retirement of Azure basic load balancer by 30 September 2025 and no support for new deployment by Mar 31, 2025. For more information, see [Azure Basic Load Balancer retires on 30 September 2025. Upgrade to Standard Load Balancer](https://azure.microsoft.com/updates/azure-basic-load-balancer-will-be-retired-on-30-september-2025-upgrade-to-standard-load-balancer).
1414

1515
## Impact on HDInsight clusters
1616

17-
### Scenario 1: Long running clusters (when you don’t create and delete cluster daily for your use case)
17+
### Long running clusters (when you don’t create and delete cluster daily for your use case)
1818

19-
**Case 1:** When HDInsight clusters are already created without custom virtual network. (No virtual network provided during cluster creation.)
19+
* **Scenario 1:** Existing HDInsight clusters without custom virtual network. (No virtual network provided during cluster creation.)
2020

21-
No immediate disruption until Mar 31, 2025. However, we strongly recommend re-creating the cluster before Mar 31, 2025 to avoid potential disruption.
21+
* No immediate disruption until Mar 31, 2025. However, we strongly recommend re-creating the cluster before Mar 31, 2025 to avoid potential disruption.
2222

23-
**Case 2:** When HDInsight clusters are already created with custom virtual network, no immediate disruption until Mar 31, 2025. However, we strongly recommend re-creating the cluster by Mar 31, 2025 to avoid potential disruption.
23+
* **Scenario 2:** Existing HDInsight clusters with custom virtual network.
2424

25-
### Scenario 2: New clusters creation
25+
* No immediate disruption until Mar 31, 2025. However, we strongly recommend re-creating the cluster before Mar 31, 2025 to avoid potential disruption.
2626

27-
**Case 1:** When HDInsight clusters already created without custom virtual network. (No virtual network provided during cluster creation.)
27+
### New cluster creation
2828

29-
In this case, no impact. You can recreate the cluster directly.
29+
* **Scenario 1:** HDInsight clusters without custom virtual network. (No virtual network provided during cluster creation)
3030

31-
**Case 2:** When HDInsight clusters are created with custom virtual network
31+
* In this case, no impact. You can recreate the cluster directly.
3232

33-
In this case, there are two options to create a cluster
34-
35-
**Approach 1:** Create the cluster with a new subnet
33+
* **Scenario 2:** HDInsight clusters with custom virtual network. (Providing your own VNet during cluster creation)
34+
35+
* In this case, there are two options to create a cluster
3636

37-
1. Choose the outbound connectivity for your cluster, and follow this document [Use Source Network Address Translation (SNAT) for outbound connections](/azure/load-balancer/load-balancer-outbound-connections), and choose one method to provide outbound connectivity for the new cluster. The most recommended way is to attach a NAT gateway and a Network Security Group (NSG) to the subnet.
37+
**Approach 1:** Create the cluster with a new subnet
3838

39-
1. (Optional) Create a new NAT gateway and a new network security group (NSG).
39+
1. Choose the outbound connectivity for your cluster, and follow this document [Use Source Network Address Translation (SNAT) for outbound connections](/azure/load-balancer/load-balancer-outbound-connections), and choose one method to provide outbound connectivity for the new cluster. The most recommended way is to attach a NAT gateway and a Network Security Group (NSG) to the subnet.
40+
41+
1. (Optional) Create a new NAT gateway and a new network security group (NSG).
4042

41-
> [!NOTE]
42-
> You can associate an existing NAT gateway and NSG or the newly created ones.
43-
>
44-
> Follow the docs [Quickstart: Create a NAT gateway - Azure portal](/azure/nat-gateway/quickstart-create-nat-gateway-portal), and [Create, change, or delete an Azure network security group](/azure/virtual-network/manage-network-security-group?tabs=network-security-group-portal#create-a-network-security-group) to create a new NAT gateway and a new network security group.
45-
>
46-
> You can refer to this document [Control network traffic in Azure HDInsight](./control-network-traffic.md#hdinsight-with-network-security-groups) to set up correct NSG rules.
43+
> [!NOTE]
44+
> You can associate an existing NAT gateway and NSG or the newly created ones.
45+
>
46+
> Follow the docs [Quickstart: Create a NAT gateway - Azure portal](/azure/nat-gateway/quickstart-create-nat-gateway-portal), and [Create, change, or delete an Azure network security group](/azure/virtual-network/manage-network-security-group?tabs=network-security-group-portal#create-a-network-security-group) to create a new NAT gateway and a new network security group.
47+
>
48+
> You can refer to this document [Control network traffic in Azure HDInsight](./control-network-traffic.md#hdinsight-with-network-security-groups) to set up correct NSG rules.
4749

48-
1. Create a new subnet and associate the subnet with the NAT gateway and network security group.
50+
1. Create a new subnet and associate the subnet with the NAT gateway and network security group.
4951

50-
:::image type="content" source="./media/load-balancer-migration-guidelines/create-subnet.png" alt-text="Screenshot showing how to create a subnet." border="true" lightbox="./media/load-balancer-migration-guidelines/create-subnet.png":::
51-
52-
> [!NOTE]
53-
> If you are unable to find the NAT gateway, see, [FAQ of the NAT gateway (Azure NAT Gateway frequently asked questions](/azure/nat-gateway/faq#are-basic-sku-resources--basic-load-balancer-and-basic-public-ip-addresses--compatible-with-a-nat-gateway).
52+
:::image type="content" source="./media/load-balancer-migration-guidelines/create-subnet.png" alt-text="Screenshot showing how to create a subnet." border="true" lightbox="./media/load-balancer-migration-guidelines/create-subnet.png":::
53+
54+
> [!NOTE]
55+
> If you are unable to find the NAT gateway, see, [FAQ of the NAT gateway (Azure NAT Gateway frequently asked questions](/azure/nat-gateway/faq#are-basic-sku-resources--basic-load-balancer-and-basic-public-ip-addresses--compatible-with-a-nat-gateway).
5456
55-
1. Create a new cluster with the subnet.
57+
1. Create a new cluster with the subnet.
5658

57-
:::image type="content" source="./media/load-balancer-migration-guidelines/create-hdinsight-cluster.png" alt-text="Screenshot showing how to create a HDInsight cluster." border="true" lightbox="./media/load-balancer-migration-guidelines/create-hdinsight-cluster.png":::
59+
:::image type="content" source="./media/load-balancer-migration-guidelines/create-hdinsight-cluster.png" alt-text="Screenshot showing how to create a HDInsight cluster." border="true" lightbox="./media/load-balancer-migration-guidelines/create-hdinsight-cluster.png":::
5860

59-
**Approach 2: Use the existing subnet**
61+
**Approach 2: Use the existing subnet**
6062

61-
To upgrade your existing custom virtual network to integrate Azure standard load balancer (which is HDInsight clusters use by default), see, [Use Source Network Address Translation (SNAT) for outbound connections](/azure/load-balancer/load-balancer-outbound-connections) to provide outbound connectivity for the cluster. The most recommended way is to attach a network security group and a NAT gateway to the subnet. Since the existing subnet which has HDInsight clusters with Azure basic load balancers can't be associated with an NAT gateway due to incompatibility with basic load balancer, there are two scenarios:
63+
- To upgrade your existing custom virtual network to integrate with Azure standard load balancer (which HDInsight clusters use by default now), see, [Use Source Network Address Translation (SNAT) for outbound connections](/azure/load-balancer/load-balancer-outbound-connections) to provide outbound connectivity for the cluster.
6264

63-
### Scenario 1: Existing subnet has no HDInsight clusters with Azure basic load balancer
65+
The most recommended way is to attach a network security group and a NAT gateway to the subnet. Since the existing subnet which has HDInsight clusters with Azure basic load balancers can't be associated with an NAT gateway due to incompatibility with basic load balancer, there are two scenarios:
6466

65-
1. (Optional) Create a new NAT gateway and a new network security group.
66-
67-
> [!NOTE]
68-
> You could associate an existing NAT gateway and NSG or the newly created ones.
69-
> Follow this document [Quickstart: Create a NAT gateway - Azure portal](/azure/nat-gateway/quickstart-create-nat-gateway-portal), and [Create, change, or delete an Azure network security group](/azure/virtual-network/manage-network-security-group?tabs=network-security-group-portal#create-a-network-security-group) to create a new NAT gateway and a new network security group.
70-
>
71-
> You can refer to this document [Control network traffic in Azure HDInsight](./control-network-traffic.md#hdinsight-with-network-security-groups) to set up correct NSG rules.
72-
73-
1. Associate the NAT gateway with your subnet along with a network security group.
74-
75-
:::image type="content" source="./media/load-balancer-migration-guidelines/associate-gateway.png" alt-text="Screenshot showing how to associate gateway." border="true" lightbox="./media/load-balancer-migration-guidelines/associate-gateway.png":::
67+
#### Case 1: Existing subnet has no HDInsight clusters with Azure Basic load balancers
68+
Follow these steps below:
7669

77-
1. Create the cluster with the subnet.
70+
1. Create a new NAT gateway and a new network security group or use the existing ones.
71+
72+
> [!NOTE]
73+
> You could associate an existing NAT gateway and NSG or the newly created ones.
74+
> Follow this document [Quickstart: Create a NAT gateway - Azure portal](/azure/nat-gateway/quickstart-create-nat-gateway-portal), and [Create, change, or delete an Azure network security group](/azure/virtual-network/manage-network-security-group?tabs=network-security-group-portal#create-a-network-security-group) to create a new NAT gateway and a new network security group.
75+
>
76+
> You can refer to this document [Control network traffic in Azure HDInsight](./control-network-traffic.md#hdinsight-with-network-security-groups) to set up correct NSG rules.
7877

79-
80-
:::image type="content" source="./media/load-balancer-migration-guidelines/security-networking.png" alt-text="Screenshot showing security networking tab." border="true" lightbox="./media/load-balancer-migration-guidelines/security-networking.png":::
78+
1. Associate the NAT gateway with your subnet along with a network security group.
8179

82-
> [!NOTE]
83-
> If you are unable to find the NAT gateway, see, [FAQ of the NAT gateway (Azure NAT Gateway frequently asked questions](/azure/nat-gateway/faq#are-basic-sku-resources--basic-load-balancer-and-basic-public-ip-addresses--compatible-with-a-nat-gateway).
80+
:::image type="content" source="./media/load-balancer-migration-guidelines/associate-gateway.png" alt-text="Screenshot showing how to associate gateway." border="true" lightbox="./media/load-balancer-migration-guidelines/associate-gateway.png":::
8481

85-
### Scenario 2: The existing subnet has HDInsight clusters with basic load balancers
82+
1. Create the cluster with the subnet.
83+
84+
:::image type="content" source="./media/load-balancer-migration-guidelines/security-networking.png" alt-text="Screenshot showing security networking tab." border="true" lightbox="./media/load-balancer-migration-guidelines/security-networking.png":::
85+
86+
> [!NOTE]
87+
> If you are unable to find the NAT gateway, see, [FAQ of the NAT gateway (Azure NAT Gateway frequently asked questions](/azure/nat-gateway/faq#are-basic-sku-resources--basic-load-balancer-and-basic-public-ip-addresses--compatible-with-a-nat-gateway).
8688

87-
**Approach 1:** The most recommended way is to associate a NAT gateway to the subnet along with network security group.
89+
#### Case 2: Existing subnet has HDInsight clusters with Azure Basic load balancers
90+
Consider one of these methods:
8891

89-
According to [Azure NAT Gateway frequently asked questions](/azure/nat-gateway/faq#are-basic-sku-resources--basic-load-balancer-and-basic-public-ip-addresses--compatible-with-a-nat-gateway), the subnet with NAT gateway and Azure standard load balancer isn't compatible.
92+
* **Approach 1:** The most recommended way is to associate a NAT gateway to the subnet along with network security group.
9093

91-
To associate with a NAT Gateway, perform the following steps.
94+
According to [Azure NAT Gateway frequently asked questions](/azure/nat-gateway/faq#are-basic-sku-resources--basic-load-balancer-and-basic-public-ip-addresses--compatible-with-a-nat-gateway), the subnet with NAT gateway and Azure standard load balancer isn't compatible.
9295

93-
1. Delete all the existing HDInsight clusters with Azure basic load balancers in this subnet.
96+
To associate with a NAT Gateway, perform the following steps.
9497

95-
1. (Optional) Create a new NAT gateway and a new network security group.
98+
1. Delete all the existing HDInsight clusters with Azure basic load balancers in this subnet.
9699

97-
> [!NOTE]
98-
> You could associate an existing NAT gateway and NSG or the newly created ones.
99-
> Follow this document [Quickstart: Create a NAT gateway - Azure portal](/azure/nat-gateway/quickstart-create-nat-gateway-portal), and [Create, change, or delete an Azure network security group](/azure/virtual-network/manage-network-security-group?tabs=network-security-group-portal#create-a-network-security-group) to create a new NAT gateway and a new network security group.
100-
>
101-
> You can refer to this document [Control network traffic in Azure HDInsight](./control-network-traffic.md#hdinsight-with-network-security-groups) to set up correct NSG rules.
100+
1. Create a new NAT gateway and a new network security group or use the existing ones.
102101

103-
1. Associate the subnet with the NAT gateway and network security group.
102+
> [!NOTE]
103+
> You could associate an existing NAT gateway and NSG or use the newly created ones.
104+
> Follow this document [Quickstart: Create a NAT gateway - Azure portal](/azure/nat-gateway/quickstart-create-nat-gateway-portal), and [Create, change, or delete an Azure network security group](/azure/virtual-network/manage-network-security-group?tabs=network-security-group-portal#create-a-network-security-group) to create a new NAT gateway and a new network security group.
105+
>
106+
> You can refer to this document [Control network traffic in Azure HDInsight](./control-network-traffic.md#hdinsight-with-network-security-groups) to set up correct NSG rules.
104107

105-
106-
:::image type="content" source="./media/load-balancer-migration-guidelines/add-subnet.png" alt-text="Screenshot showing how to add subnet." border="true" lightbox="./media/load-balancer-migration-guidelines/add-subnet.png":::
108+
1. Associate the subnet with the NAT gateway and network security group.
107109

108-
> [!NOTE]
109-
> If you are unable to find the NAT gateway, see, [FAQ of the NAT gateway (Azure NAT Gateway frequently asked questions](/azure/nat-gateway/faq#are-basic-sku-resources--basic-load-balancer-and-basic-public-ip-addresses--compatible-with-a-nat-gateway).
110+
:::image type="content" source="./media/load-balancer-migration-guidelines/add-subnet.png" alt-text="Screenshot showing how to add subnet." border="true" lightbox="./media/load-balancer-migration-guidelines/add-subnet.png":::
111+
112+
> [!NOTE]
113+
> If you are unable to find the NAT gateway, see, [FAQ of the NAT gateway (Azure NAT Gateway frequently asked questions](/azure/nat-gateway/faq#are-basic-sku-resources--basic-load-balancer-and-basic-public-ip-addresses--compatible-with-a-nat-gateway).
110114

111-
1. Re-create the clusters with the subnet.
115+
1. Re-create the clusters with the subnet.
112116

113-
114-
:::image type="content" source="./media/load-balancer-migration-guidelines/virtual-network.png" alt-text="Screenshot showing virtual network." border="true" lightbox="./media/load-balancer-migration-guidelines/virtual-network.png":::
117+
:::image type="content" source="./media/load-balancer-migration-guidelines/virtual-network.png" alt-text="Screenshot showing virtual network." border="true" lightbox="./media/load-balancer-migration-guidelines/virtual-network.png":::
115118

116-
**Approach 2:** Select option other than **Associate a NAT gateway to the subnet** provided in [Use Source Network Address Translation (SNAT) for outbound connections](/azure/load-balancer/load-balancer-outbound-connections), and follow the instruction for the selected option.
119+
* **Approach 2:** Select option other than **Associate a NAT gateway to the subnet** provided in [Use Source Network Address Translation (SNAT) for outbound connections](/azure/load-balancer/load-balancer-outbound-connections), and follow the instruction for the selected option.
117120

118-
**Approach 3:** Create a new subnet and then create the cluster with the new subnet.
121+
* **Approach 3:** Create a new subnet and then create the cluster with the new subnet.

0 commit comments

Comments
 (0)