Skip to content

Commit b76aa07

Browse files
authored
Update azure-vmware-solution-nsx-scale-and-performance-recommendations-for-vmware-hcx.md
Removed any unnecessary white space, formatted images and bullet points to our standard. Added a descriptive sentence after any heading.
1 parent 9c11b9c commit b76aa07

File tree

1 file changed

+25
-47
lines changed

1 file changed

+25
-47
lines changed

articles/azure-vmware/azure-vmware-solution-nsx-scale-and-performance-recommendations-for-vmware-hcx.md

Lines changed: 25 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -7,61 +7,57 @@ ms.date: 12/18/2024
77
ms.custom: engagement-fy25
88
---
99

10-
1110
# NSX Scale and performance recommendations for VMware HCX
1211

13-
14-
1512
In this article, learn about the default NSX topology in Azure VMware Solution, NSX data path performance characteristics, how to identify NSX data path resource constraints and recommended configurations to help mitigate resource constraints and optimize over all data path performance for HCX migrations.
1613

1714
## Azure VMware Solution NSX Default topology
1815

1916
The Azure VMware Solution NSX default topology has the following configuration:
2017

21-
• Three node NSX Manager cluster.
22-
23-
• NSX Edge and Gateway for North-bound traffic:
18+
* Three node NSX Manager cluster.
2419

25-
• Two Large Form Factor NSX Edges, deployed in an NSX Edge cluster.
20+
* NSX Edge and Gateway for North-bound traffic:
2621

27-
• A Default NSX Tier-0 Gateway in Active/Active mode.
28-
29-
• A Default NSX Tier-1 Gateway in Active/Standby mode.
30-
31-
• A Default HCX-UPLINK segment connected to default Tier-1 Gateway.
22+
* Two Large Form Factor NSX Edges, deployed in an NSX Edge cluster.
3223

24+
* A Default NSX Tier-0 Gateway in Active/Active mode.
3325

26+
* A Default NSX Tier-1 Gateway in Active/Standby mode.
3427

28+
* A Default HCX-UPLINK segment connected to default Tier-1 Gateway.
29+
3530
Customers typically host their application workloads by creating new NSX segments and attaching them to the default Tier-1 Gateway. Additionally, customers with an HCX migration use case use the default HCX-uplink segment, which is also connected to the default Tier-1 Gateway.
3631

37-
The default NSX topology for Azure VMware Solution, where all traffic exits through the default Tier-1 Gateway, may not be optimal based on customer traffic flows and throughput requirements. Here are some potential challenges and the recommended configurations to optimize the NSX Edge data path resource.
32+
The default NSX topology for Azure VMware Solution, where all traffic exits through the default Tier-1 Gateway, may not be optimal based on customer traffic flows and throughput requirements.
3833

3934
### Potential Challenge:
4035

41-
• All the north-bound network traffic (Migrations, L2 Extensions, VM traffic outbound of Azure VMware Solution) uses the default Tier-1 Gateway which is in Active/Standby mode.
36+
Here are some potential challenges and the recommended configurations to optimize the NSX Edge data path resource.
4237

43-
• In the default Active/Standby mode, the Tier-1 Gateway only uses the Active Edge VM for all north-bound traffic.
38+
* All the north-bound network traffic (Migrations, L2 Extensions, VM traffic outbound of Azure VMware Solution) uses the default Tier-1 Gateway which is in Active/Standby mode.
4439

45-
• The second Edge VM, which is standby, is not used for north-bound traffic.
40+
* In the default Active/Standby mode, the Tier-1 Gateway only uses the Active Edge VM for all north-bound traffic.
4641

47-
• Depending on the throughput requirements, and flows this could potentially create a bottleneck on the Active Edge VM.
42+
* The second Edge VM, which is standby, is not used for north-bound traffic.
43+
44+
* Depending on the throughput requirements, and flows this could potentially create a bottleneck on the Active Edge VM.
4845

4946
### Recommended Practices:
5047
It is possible to change the NSX North-bound network connectivity to distribute the traffic evenly to both Edge VMs. Creating an additional Tier-1 Gateways and distributing the NSX segments across multiple Tier-1 Gateways evenly distributes traffic across the Edge VMs. For an HCX migration use case, the recommendation would be to move HCX Layer 2 (L2) Extension and migration traffic to a newly created Tier-1 Gateway, so it uses the NSX Edge resource optimally.
5148

5249
To make an Active Edge for a given Tier-1 Gateway predictable, it is recommended to create an additional Tier-1 Gateway with the High Availability (HA) Mode set to Active/Standby with the Failover mode set to preemptive. This configuration allows you to select a different active Edge VM than the one in use by the default Tier-1 Gateway. This naturally splits north-bound traffic across multiple Tier-1 Gateways, so both NSX Edges are optimally utilized, thus avoiding a potential bottleneck with the default NSX topology.
53-
54-
:::image type="content" source="media/nsxt/default-nsx-topology.png" alt-text="Diagram showing the default nsx topology in Azure VMware Solution." border="false" lightbox="media/nsxt/default-nsx-topology.png":::
55-
5650

57-
Figure 1: Depicts the default NSX topology in Azure VMware Solution
51+
:::image type="content" source="media/nsxt/default-nsx-topology.png" alt-text="Diagram showing the default nsx topology in Azure VMware Solution." border="false" lightbox="media/nsxt/default-nsx-topology.png":::
5852

5953
### NSX Edge performance characteristics
6054

6155
Each of the NSX Edge Virtual machine (EVM) can support up to approximately ~20 Gbps based on the number of flows, packet size, and services enabled on the NSX gateways. Each of the Edge VMs (Large form factors) has four Data Plane Development Kit (DPDK) enabled CPU cores, essentially each of the DPDK core could process up to ~5 Gbps traffic, based on flow hashing, packet size, and services enabled on NSX gateway. For more information on NSX Edge performance, see the VMware NSX-T Reference Design Guide section 8.6.2.*
6256

6357
## Monitor, Identify, and Fix potential Edge data path Performance Bottlenecks
6458

59+
Using the built-in NSX alarm framework is recommended to monitor and identify key NSX edge performance metrics.
60+
6561
### How to Monitor and Identify NSX Edge Data Path Resource Constraints:
6662

6763
NSX Edge performance can be monitored and identified by using the built-in NSX alarm framework. The following critical NSX Edge alarms identify the NSX Edge data path resource constraints:
@@ -73,37 +69,33 @@ NSX Edge performance can be monitored and identified by using the built-in NSX a
7369
3. Edge Datapath NIC throughput high.
7470

7571
:::image type="content" source="media/nsxt/nsx-edge-critical-alerts.png" alt-text="Diagram showing nsx edge health critical alerts." border="false" lightbox="media/nsxt/nsx-edge-critical-alerts.png":::
76-
77-
Figure 2: NSX Edge Health Critical Alerts
7872

7973
## How to fix the NSX Edge resource constraints:
8074

8175
To validate the issue, check the historic and real-time traffic throughput:
8276

8377
Validation of issue:
8478

85-
Historic/Realtime traffic throughput – check traffic throughput at the alarm time for the correlation.
79+
* Historic/Realtime traffic throughput – check traffic throughput at the alarm time for the correlation.
8680

8781
:::image type="content" source="media/nsxt/nsx-edge-performance-charts.png" alt-text="Diagram showing nsx edge vm performance charts." border="false" lightbox="media/nsxt/nsx-edge-performance-charts.png":::
8882

89-
Figure 3: NSX Edge VM Performance Charts
90-
9183
To mitigate the issue, here are a few options to consider.
9284

9385
Mitigation options:
9486
1. Edge Scale-UP: NSX Edge Scale-UP from Large (four DPDK CPU) to X-Large (eight DPDK CPU) form factor could resolve part of the issue.
9587

96-
Edge Scale up provides additional CPU and memory for data path packet processing.
88+
* Edge Scale up provides additional CPU and memory for data path packet processing.
9789

98-
Edge Scale up may not help if you have one or more heavy flows, for example, HCX Network Extension (NE) to Network Extension (NE) traffic, as this traffic could potentially pin to one of the DPDK CPU cores.
90+
* Edge Scale up may not help if you have one or more heavy flows, for example, HCX Network Extension (NE) to Network Extension (NE) traffic, as this traffic could potentially pin to one of the DPDK CPU cores.
9991

10092
2. Tier-1 Gateway Topology Change: Change the Azure VMware Solution NSX default Tier-1 Gateway topology with multiple Tier-1 Gateways to split the traffic across multiple Edge VMs
10193

102-
More details in the next section with an example of HCX migration use case.
94+
* More details in the next section with an example of HCX migration use case.
10395

10496
3. Edge Scale-OUT: If customer has large number of Hosts in the SDDC and workloads, NSX Edge Scale-OUT (from two Edges to four Edges) could be an option to add additional NSX Edge data path resources.
10597

106-
However, NSX Edge Scale-OUT is effective only with a change in the NSX default Tier-1 Gateway topology to distribute the traffic optimally across all four Edge VMs. More details in the next section with an example of HCX migration use case.
98+
* However, NSX Edge Scale-OUT is effective only with a change in the NSX default Tier-1 Gateway topology to distribute the traffic optimally across all four Edge VMs. More details in the next section with an example of HCX migration use case.
10799

108100
### Default and configuration recommendations to the NSX Edge data path performance.
109101

@@ -133,14 +125,14 @@ In general, creating additional Tier-1 Gateways and distributing segments across
133125

134126
:::image type="content" source="media/nsxt/nsx-traffic-flow-additional-tier-1-gateway.png" alt-text="Diagram showing nsx traffic flow in Azure VMware Solution with an additional Tier-1 gateway." border="false" lightbox="media/nsxt/nsx-traffic-flow-additional-tier-1-gateway.png":::
135127

136-
Figure 4: NSX traffic Flow with additional Tier-1 Gateways created.
137-
138-
139128
### Detailed Steps (Mitigate Edge VM bottleneck)
140129

130+
The creation of an additional Tier-1 Gateway can help mitigate potential Edge VM bottlenecks.
131+
141132
[Create an NSX Tier-1 Gateway](tutorial-nsx-tier-1-gateway.md).
142133

143134
Distributed Only Option:
135+
144136
1. No Edge Cluster can be selected.
145137

146138
2. All connected Segments and Service Ports must be advertised.
@@ -149,12 +141,9 @@ Distributed Only Option:
149141

150142
:::image type="content" source="media/nsxt/nsx-tier-1-gateway-distributed-only.png" alt-text="Diagram showing nsx tier-1 gateway distributed only option." border="false" lightbox="media/nsxt/nsx-tier-1-gateway-distributed-only.png":::
151143

152-
Figure 5: Tier-1 Gateway Distributed Only Option.
153-
154144
>[!IMPORTANT]
155145
>In a Distributed Only High Availability (HA) Mode, traffic is distributed across all Edge VMs. Workload traffic and Migration traffic may traverse the Active Edge at the same time.
156146
157-
158147
Active/Standby Option:
159148

160149
1. Select the **Edge Cluster**.
@@ -173,12 +162,9 @@ An Active/Standby configuration with the preferred Edge VM defined allows you to
173162

174163
:::image type="content" source="media/nsxt/nsx-tier-1-gateway-active-standby.png" alt-text="Diagram showing nsx tier-1 gateway active standby option." border="false" lightbox="media/nsxt/nsx-tier-1-gateway-active-standby.png":::
175164

176-
Figure 6: Tier-1 Gateway Active/Standby Option.
177-
178165
>[!NOTE]
179166
>Microsoft Recommends the Active/Standby HA Mode when additional Tier-1 Gateways are created. This allows customers to seperate Workload and migration traffic across different Edge VMs.
180167
181-
182168
## Create a new Segment for HCX Uplink and attach to the new Tier-1 Gateway
183169

184170
For detailed instructions on NSX Segment creation. [NSX Segment Creation](tutorial-nsx-t-network-segment.md)
@@ -190,8 +176,6 @@ Select the newly created Tier-1 Gateway when creating your new NSX Segment.
190176
191177
:::image type="content" source="media/nsxt/nsx-segment-creation.png" alt-text="Diagram showing the creation of a nsx segment." border="false" lightbox="media/nsxt/nsx-segment-creation.png":::
192178

193-
Figure 7: NSX Segment creation for new HCX Uplink network.
194-
195179
## Create an HCX Network Profile
196180

197181
For detailed steps on how to Create an HCX Network Profile. [HCX Network Profile](configure-vmware-hcx.md#create-network-profiles)
@@ -210,23 +194,17 @@ For detailed steps on how to Create an HCX Network Profile. [HCX Network Profile
210194

211195
:::image type="content" source="media/hcx/hcx-uplink-network-profile.png" alt-text="Diagram showing the creation of a hcx network profile." border="false" lightbox="media/nsxt/hcx-uplink-network-profile.png":::
212196

213-
Figure 8: HCX Network profile creation.
214-
215197
Once the new HCX Uplink Network Profile is created, update the existing Service Mesh and edit the default uplink profile with the newly created Network Profile.
216198

217199
:::image type="content" source="media/hcx/hcx-service-mesh-edit.png" alt-text="Diagram showing how to edit an existing HCX service mesh." border="false" lightbox="media/nsxt/hcx-service-mesh-edit.png":::
218200

219-
Figure 9: HCX Service Mesh edit.
220-
221201
7. Select the existing **Service Mesh** and select **Edit**.
222202

223203
8. Edit the default Uplink with the newly created Network Profile.
224204

225205
9. Select **Service Mesh Change**.
226206

227207
:::image type="content" source="media/hcx/hcx-in-service-mode.png" alt-text="Diagram showing how to edit an in service mode on a hcx Network extension appliance." border="false" lightbox="media/nsxt/hcx-in-service-mode.png":::
228-
229-
Figure 10: HCX In-Service Mode
230208

231209
>[!Note]
232210
>In-Service Mode of the HCX Network Extension appliances should be considered to reduce downtime during this Service Mesh edit.

0 commit comments

Comments
 (0)