Skip to content

Commit 6d20c47

Browse files
authored
Merge pull request #224289 from ShawnJackson/concepts-network-design-considerations
[AQ] edit pass: concepts-network-design-considerations
2 parents 8bc747e + 84ff019 commit 6d20c47

File tree

1 file changed

+44
-47
lines changed

1 file changed

+44
-47
lines changed

articles/azure-vmware/concepts-network-design-considerations.md

Lines changed: 44 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -8,112 +8,109 @@ ms.date: 1/10/2023
88

99
# Azure VMware Solution network design considerations
1010

11-
Azure VMware Solution offers a VMware private cloud environment accessible for users and applications from on-premises and Azure-based environments or resources. The connectivity is delivered through networking services such as Azure ExpressRoute and VPN connections. There are several networking considerations to review before setting up your Azure VMware Solution environment. This article provides solutions for use cases you may encounter when configuring your networking with Azure VMware Solution.
11+
Azure VMware Solution offers a VMware private cloud environment that users and applications can access from on-premises and Azure-based environments or resources. Networking services such as Azure ExpressRoute and virtual private network (VPN) connections deliver the connectivity.
12+
13+
There are several networking considerations to review before you set up your Azure VMware Solution environment. This article provides solutions for use cases that you might encounter when you're using Azure VMware Solution to configure your networks.
1214

1315
## Azure VMware Solution compatibility with AS-Path Prepend
1416

15-
Azure VMware Solution is compatible with AS-Path Prepend for redundant ExpressRoute configurations with the caveat of not honoring the outbound path selection from Azure towards on-premises. If you're running two or more ExpressRoute paths between on-premises and Azure, and the listed [Prerequisites](#prerequisites) are not met, you may experience impaired connectivity or no connectivity between your on-premises networks and Azure VMware Solution. The connectivity issue is caused when Azure VMware Solution doesn't see the AS-Path Prepend and uses equal cost multi-pathing (ECMP) to send traffic towards your environment over both ExpressRoute circuits. That action causes issues with stateful firewall inspection.
17+
Azure VMware Solution is compatible with AS-Path Prepend for redundant ExpressRoute configurations, with the caveat of not honoring the outbound path selection from Azure toward on-premises. If you're running two or more ExpressRoute paths between on-premises and Azure, and you don't meet the listed [prerequisites](#prerequisites), you might experience impaired connectivity or no connectivity between your on-premises networks and Azure VMware Solution.
18+
19+
The connectivity problem happens when Azure VMware Solution doesn't notice AS-Path Prepend and uses equal-cost multipath (ECMP) routing to send traffic toward your environment over both ExpressRoute circuits. That action causes problems with stateful firewall inspection.
1620

1721
### Prerequisites
1822

19-
For AS-Path Prepend, you'll need to verify that all of the following listed connections are true:
23+
For AS-Path Prepend, verify that all of the following listed connections are true:
2024

2125
> [!div class="checklist"]
22-
> * Both or all circuits are connected to Azure VMware Solution with ExpressRoute Global Reach.
26+
> * Both or all circuits are connected to Azure VMware Solution through ExpressRoute Global Reach.
2327
> * The same netblocks are being advertised from two or more circuits.
2428
> * Stateful firewalls are in the network path.
2529
> * You're using AS-Path Prepend to force Azure to prefer one path over others.
2630
27-
Either 2 or 4 byte Public ASN numbers should be used and be compatible with Azure VMware Solution. If you don't own a Public ASN to use for prepending, open a [Microsoft Customer Support Ticket](https://ms.portal.azure.com/#view/Microsoft_Azure_Support/HelpAndSupportBlade/~/overview) to view options.
31+
Use either 2-byte or 4-byte public ASN numbers, and make sure that they're compatible with Azure VMware Solution. If you don't own a public ASN for prepending, open a [Microsoft support ticket](https://ms.portal.azure.com/#view/Microsoft_Azure_Support/HelpAndSupportBlade/~/overview) to view options.
2832

2933
## Management VMs and default routes from on-premises
3034

3135
> [!IMPORTANT]
32-
> Azure VMware Solution Management VMs will not honor a default route from on-premises.
36+
> Azure VMware Solution management virtual machines (VMs) won't honor a default route from on-premises.
3337
34-
If you're routing back to your on-premises networks using only a default route advertised towards Azure, the vCenter Server and NSX-T Manager VMs won't be compatible with that route.
38+
If you're routing back to your on-premises networks by using only a default route advertised toward Azure, vCenter Server and NSX-T Manager VMs won't be compatible with that route.
3539

36-
**Solution**
40+
To reach vCenter Server and NSX-T Manager, provide specific routes from on-premises to allow traffic to have a return path to those networks.
3741

38-
To reach vCenter Server and NSX-T Manager, more specific routes from on-premises need to be provided to allow traffic to have a return path route to those networks.
42+
## Default route to Azure VMware Solution for internet traffic inspection
3943

40-
## Use a default route to Azure VMware Solution for internet traffic inspection
44+
Certain deployments require inspecting all egress traffic from Azure VMware Solution toward the internet. Although it's possible to create network virtual appliances (NVAs) in Azure VMware Solution, there are use cases where these appliances already exist in Azure and can be applied to inspect internet traffic from Azure VMware Solution. In this case, a default route can be injected from the NVA in Azure to attract traffic from Azure VMware Solution and inspect the traffic before it goes out to the public internet.
4145

42-
Certain deployments require inspecting all egress traffic from Azure VMware Solution towards the Internet. While it's possible to create Network Virtual Appliances (NVAs) in Azure VMware Solution, there are use cases when these appliances already exist in Azure that can be applied to inspect Internet traffic from Azure VMware Solution. In this case, a default route can be injected from the NVA in Azure to attract traffic from Azure VMware Solution and inspect it before sending it out to the public Internet.
46+
The following diagram describes a basic hub-and-spoke topology connected to an Azure VMware Solution cloud and to an on-premises network through ExpressRoute. The diagram shows how the NVA in Azure originates the default route (`0.0.0.0/0`). Azure Route Server propagates the route to Azure VMware Solution through ExpressRoute.
4347

44-
The following diagram describes a basic hub and spoke topology connected to an Azure VMware Solution cloud and to an on-premises network through ExpressRoute. The diagram shows how the default route (`0.0.0.0/0`) is originated by the NVA in Azure, and propagated by Azure Route Server to Azure VMware Solution through ExpressRoute.
45-
46-
:::image type="content" source="media/concepts-network-design/vmware-solution-default.png" alt-text="Diagram of Azure VMware Solution with Route Server and default route." lightbox="media/concepts-network-design/vmware-solution-default.png":::
48+
:::image type="content" source="media/concepts-network-design/vmware-solution-default.png" alt-text="Diagram of Azure VMware Solution with Route Server and a default route." lightbox="media/concepts-network-design/vmware-solution-default.png":::
4749

4850
> [!IMPORTANT]
49-
> The default route advertised by the NVA will be propagated to the on-premises network. Because of that, UDRs will need to be added to ensure traffic from Azure VMware Solution is transiting through the NVA.
51+
> The default route that the NVA advertises will be propagated to the on-premises network. You need to add user-defined routes (UDRs) to ensure that traffic from Azure VMware Solution is transiting through the NVA.
5052
5153
Communication between Azure VMware Solution and the on-premises network usually occurs over ExpressRoute Global Reach, as described in [Peer on-premises environments to Azure VMware Solution](../azure-vmware/tutorial-expressroute-global-reach-private-cloud.md).
5254

53-
## Connectivity between Azure VMware Solution and on-premises network via a third party network virtual appliance
55+
## Connectivity between Azure VMware Solution and an on-premises network
5456

55-
There are two main scenarios for this connectivity pattern:
57+
There are two main scenarios for connectivity between Azure VMware Solution and an on-premises network via a third-party NVA:
5658

57-
- Organizations may have the requirement to send traffic between Azure VMware Solution and the on-premises network through an NVA (typically a firewall).
58-
- ExpressRoute Global Reach might not be available in a particular region to interconnect the ExpressRoute circuits of Azure VMware Solution and the on-premises network.
59+
- Organizations have a requirement to send traffic between Azure VMware Solution and the on-premises network through an NVA (typically a firewall).
60+
- ExpressRoute Global Reach isn't available in a particular region to interconnect the ExpressRoute circuits of Azure VMware Solution and the on-premises network.
5961

60-
There are two topologies you can apply to meet all requirements for these two scenarios. The first is a [Supernet topology](#supernet-design-topology) and the second is a [Transit spoke virtual network topology](#transit-spoke-virtual-network-topology).
62+
There are two topologies that you can apply to meet all requirements for those scenarios: [supernet](#supernet-design-topology) and [transit spoke virtual network](#transit-spoke-virtual-network-topology).
6163

6264
> [!IMPORTANT]
63-
> The preferred option to connect Azure VMware Solution and on-premises environments is a direct ExpressRoute Global Reach connection. The patterns described in this document add considerable complexity to the environment.
65+
> The preferred option to connect Azure VMware Solution and on-premises environments is a direct ExpressRoute Global Reach connection. The patterns described in this article add complexity to the environment.
6466
6567
### Supernet design topology
6668

67-
If both ExpressRoute circuits (to Azure VMware Solution and to on-premises) are terminated in the same ExpressRoute gateway, you can assume that the gateway is going to route packets across them. However, an ExpressRoute gateway isn't designed to do that. You need to hairpin the traffic to an NVA that can route the traffic. There are two requirements to hairpin network traffic to an NVA:
69+
If both ExpressRoute circuits (to Azure VMware Solution and to on-premises) are terminated in the same ExpressRoute gateway, you can assume that the gateway is going to route packets across them. However, an ExpressRoute gateway isn't designed to do that. You need to hairpin the traffic to an NVA that can route the traffic.
6870

69-
- The NVA should advertise a supernet for the Azure VMware Solution and on-premises prefixes.
71+
There are two requirements to hairpin network traffic to an NVA:
7072

71-
You could use a supernet that includes both Azure VMware Solution and on-premises prefixes, or individual prefixes for Azure VMware Solution and on-premises (always less specific that the actual prefixes advertised over ExpressRoute). Keep in mind that all supernet prefixes advertised to Route Server are going to be propagated both to Azure VMware Solution and on-premises.
72-
- UDRs in the GatewaySubnet that exactly match the prefixes advertised from Azure VMware Solution and on-premises will cause hairpin traffic from the GatewaySubnet to the NVA.
73+
- The NVA should advertise a supernet for the Azure VMware Solution and on-premises prefixes.
7374

74-
**This topology results in high management overhead for large networks that change over time. Note that there are specific limitations to be considered.**
75+
You could use a supernet that includes both Azure VMware Solution and on-premises prefixes. Or you could use individual prefixes for Azure VMware Solution and on-premises (always less specific than the actual prefixes advertised over ExpressRoute). Keep in mind that all supernet prefixes advertised to Route Server will be propagated to both Azure VMware Solution and on-premises.
76+
- UDRs in the gateway subnet that exactly match the prefixes advertised from Azure VMware Solution and on-premises will cause hairpin traffic from the gateway subnet to the NVA.
7577

76-
**Limitations**
78+
This topology results in high management overhead for large networks that change over time. Consider these limitations:
7779

78-
- Anytime a workload segment is created in Azure VMware Solution, UDRs may need to be added to ensure traffic from Azure VMware Solution is transiting through the NVA.
79-
- If your on-premises environment has a large number of routes that change, BGP and UDR configuration in the supernet may need to be updated.
80-
- Since there's a single ExpressRoute Gateway that processes network traffic in both directions, performance may be limited.
80+
- Anytime a workload segment is created in Azure VMware Solution, UDRs might need to be added to ensure that traffic from Azure VMware Solution is transiting through the NVA.
81+
- If your on-premises environment has a large number of routes that change, Border Gateway Protocol (BGP) and UDR configuration in the supernet might need to be updated.
82+
- Because a single ExpressRoute gateway processes network traffic in both directions, performance might be limited.
8183
- There's an Azure Virtual Network limit of 400 UDRs.
8284

83-
The following diagram demonstrates how the NVA needs to advertise more generic (less specific) prefixes that include the networks from on-premises and Azure VMware Solution. Be careful with this approach as the NVA could potentially attract traffic that it shouldn't (since it's advertising wider ranges, for example: the whole `10.0.0.0/8` network).
85+
The following diagram demonstrates how the NVA needs to advertise prefixes that are more generic (less specific) and that include the networks from on-premises and Azure VMware Solution. Be careful with this approach. The NVA could potentially attract traffic that it shouldn't, because it's advertising wider ranges (for example, the whole `10.0.0.0/8` network).
8486

8587
:::image type="content" source="media/concepts-network-design/vmware-solution-to-on-premises-hairpin.png" alt-text="Diagram of Azure VMware Solution to on-premises communication with Route Server in a single region." lightbox="media/concepts-network-design/vmware-solution-to-on-premises-hairpin.png":::
8688

8789
### Transit spoke virtual network topology
8890

8991
> [!NOTE]
90-
> If advertising less specific prefixes is not possible due to the limits previously described, you can implement an alternative design using two separate Virtual Networks.
92+
> If advertising prefixes that are less specific isn't possible because of the previously described limits, you can implement an alternative design that uses two separate virtual networks.
9193
92-
In this topology, instead of propagating less specific routes to attract traffic to the ExpressRoute gateway, two different NVAs in separate Virtual Networks can exchange routes between each other. The Virtual Networks can propagate these routes to their respective ExpressRoute circuits via BGP and Azure Route Server, as the following diagram shows. Each NVA has full control on which prefixes are propagated to each ExpressRoute circuit.
94+
In this topology, instead of propagating routes that are less specific to attract traffic to the ExpressRoute gateway, two different NVAs in separate virtual networks can exchange routes between each other. The virtual networks can propagate these routes to their respective ExpressRoute circuits via BGP and Azure Route Server. Each NVA has full control over which prefixes are propagated to each ExpressRoute circuit.
9395

94-
The following diagram demonstrates how a single 0.0.0.0/0 is advertised to Azure VMware Solution. It also shows how the individual Azure VMware Solution prefixes are propagated to the on-premises network.
96+
The following diagram demonstrates how a single `0.0.0.0/0` route is advertised to Azure VMware Solution. It also shows how the individual Azure VMware Solution prefixes are propagated to the on-premises network.
9597

9698
:::image type="content" source="media/concepts-network-design/vmware-solution-to-on-premises.png" alt-text="Diagram of Azure VMware Solution to on-premises communication with Route Server in two regions." lightbox="media/concepts-network-design/vmware-solution-to-on-premises.png":::
9799

98100
> [!IMPORTANT]
99-
> An encapsulation protocol such as VXLAN or IPsec is required between the NVAs. Encapsulation is needed because the NVA NICs would learn the routes from Azure Route Server with the NVA as next hop and create a routing loop.
101+
> An encapsulation protocol such as VXLAN or IPsec is required between the NVAs. Encapsulation is needed because the NVA network adapter (NIC) would learn the routes from Azure Route Server with the NVA as the next hop and create a routing loop.
100102
101-
There's an alternative to using an overlay. Apply secondary NICs in the NVA that won't learn the routes from Azure Route Server and configure UDRs so that Azure can route traffic to the remote environment over those NICs. You can find more details in [Enterprise-scale network topology and connectivity for Azure VMware Solution](/azure/cloud-adoption-framework/scenarios/azure-vmware/eslz-network-topology-connectivity#scenario-2-a-third-party-nva-in-hub-azure-virtual-network-inspects-all-network-traffic).
103+
There's an alternative to using an overlay. Apply secondary NICs in the NVA that won't learn the routes from Azure Route Server. Then, configure UDRs so that Azure can route traffic to the remote environment over those NICs. You can find more details in [Enterprise-scale network topology and connectivity for Azure VMware Solution](/azure/cloud-adoption-framework/scenarios/azure-vmware/eslz-network-topology-connectivity#scenario-2-a-third-party-nva-in-hub-azure-virtual-network-inspects-all-network-traffic).
102104

103-
**This topology requires a complex initial set-up. Once the set-up is complete, the topology works as expected with minimal management overhead. See the following list of specific set-up complexities.**
105+
This topology requires a complex initial setup. The topology then works as expected with minimal management overhead. Setup complexities include:
104106

105-
- There's an extra cost for an additional transit Virtual Network that includes an Azure Route Server, ExpressRoute Gateway, and another NVA. The NVAs may also need to use large VM sizes to meet throughput requirements.
106-
- There's IPSec or VxLAN tunneling between the two NVAs required which means that the NVAs are also in the datapath. Depending on the type of NVA you're using, it can result in custom and complex configuration on those NVAs.
107+
- There's an extra cost for an additional transit virtual network that includes Azure Route Server, an ExpressRoute gateway, and another NVA. The NVAs might also need to use large VM sizes to meet throughput requirements.
108+
- IPsec or VXLAN tunneling is required between the two NVAs, which means that the NVAs are also in the datapath. Depending on the type of NVA that you're using, it can result in custom and complex configuration on those NVAs.
107109

108110
## Next steps
109111

110-
Now that you've covered Azure VMware Solution network design considerations, you may want to learn more about:
112+
Now that you've covered network design considerations for Azure VMware Solution, you might want to learn more about these topics:
111113

112-
- [Network interconnectivity concepts - Azure VMware Solution](concepts-networking.md)
114+
- [Azure VMware Solution networking and interconnectivity concepts](concepts-networking.md)
113115
- [Plan the Azure VMware Solution deployment](plan-private-cloud-deployment.md)
114-
- [Networking planning checklist for Azure VMware Solution](tutorial-network-checklist.md)
115-
116-
## Recommended content
117-
118-
- [Tutorial - Configure networking for your VMware private cloud in Azure - Azure VMware Solution](tutorial-network-checklist.md)
119-
116+
- [Tutorial: Networking planning checklist for Azure VMware Solution](tutorial-network-checklist.md)

0 commit comments

Comments
 (0)