Merge pull request #197247 from duongau/vneter

ExpressRoute - VNet to VNet guidance

Author: jborsecnik

articles/expressroute/TOC.yml

@@ -97,6 +97,8 @@
     href: expressroute-about-encryption.md
   - name: Connect Azure to public cloud
     href: expressroute-connect-azure-to-public-cloud.md
+  - name: Connectivity between virtual networks
+    href: virtual-network-connectivity-guidance.md
   - name: Cross-network connectivity
     href: cross-network-connectivity.md
   - name: BGP communities

articles/expressroute/expressroute-about-virtual-network-gateways.md

@@ -27,7 +27,7 @@ Each virtual network can have only one virtual network gateway per gateway type.
 ## <a name="gwsku"></a>Gateway SKUs
 [!INCLUDE [expressroute-gwsku-include](../../includes/expressroute-gwsku-include.md)]
 
-If you want to upgrade your gateway to a more powerful gateway SKU, you can use the 'Resize-AzVirtualNetworkGateway' PowerShell cmdlet or perform the upgrade directly in the ExpressRoute virtual network gateway configuration blade in the Azure Portal. The following upgrades are supported:
+If you want to upgrade your gateway to a more powerful gateway SKU, you can use the 'Resize-AzVirtualNetworkGateway' PowerShell cmdlet or perform the upgrade directly in the ExpressRoute virtual network gateway configuration blade in the Azure portal. The following upgrades are supported:
 
 - Standard to High Performance
 - Standard to Ultra Performance
@@ -135,6 +135,10 @@ For additional technical resources and specific syntax requirements when using R
 | [PowerShell](/powershell/module/servicemanagement/azure.service/#azure) |[PowerShell](/powershell/module/az.network#networking) |
 | [REST API](/previous-versions/azure/reference/jj154113(v=azure.100)) |[REST API](/rest/api/virtual-network/) |
 
+## VNet-to-VNet connectivity
+
+By default, connectivity between virtual networks are enabled when you link multiple virtual networks to the same ExpressRoute circuit. However, Microsoft advises against using your ExpressRoute circuit for communication between virtual networks and instead use [VNet peering](../virtual-network/virtual-network-peering-overview.md). For more information about why VNet-to-VNet connectivity is not recommended over ExpressRoute, see [connectivity between virtual networks over ExpressRoute](virtual-network-connectivity-guidance.md).
+
 ## Next steps
 
 For more information about available connection configurations, see [ExpressRoute Overview](expressroute-introduction.md).

articles/expressroute/expressroute-howto-linkvnet-portal-resource-manager.md

@@ -47,6 +47,8 @@ In this tutorial, you learn how to:
 
 * In order to create the connection from the ExpressRoute circuit to the target ExpressRoute virtual network gateway, the number of address spaces advertised from the local or peered virtual networks needs to be equal to or less than **200**. Once the connection has been successfully created, you can add additional address spaces, up to 1,000, to the local or peered virtual networks.
 
+* Review guidance for [connectivity between virtual networks over ExpressRoute](virtual-network-connectivity-guidance.md).
+
 * You can [view a video](https://azure.microsoft.com/documentation/videos/azure-expressroute-how-to-create-a-connection-between-your-vpn-gateway-and-expressroute-circuit) before beginning to better understand the steps.
 
 ## Connect a VNet to a circuit - same subscription

articles/expressroute/virtual-network-connectivity-guidance.md

@@ -0,0 +1,33 @@
+---
+title: 'Connectivity between virtual networks over ExpressRoute'
+description: This article explains why virtual network peering is the recommended solution for VNet to VNet connectivity when using ExpressRoute.
+services: expressroute
+author: duongau
+ms.service: expressroute
+ms.topic: conceptual
+ms.date: 05/05/2022
+ms.author: duau
+---
+
+# Connectivity between virtual networks over ExpressRoute
+
+## Overview
+
+ExpressRoute private peering supports connectivity between multiple virtual networks. To achieve this connectivity, an ExpressRoute virtual network gateway gets deployed into each virtual network. Then a connection is created between the gateway and the ExpressRoute circuit. When this connection gets established, connectivity to virtual machines (VMs) and private endpoints are enabled from on-premises. When multiple virtual networks are linked to an ExpressRoute circuit, VNet to VNet connectivity is enabled. Although this behavior happens by default when linking virtual networks to the same ExpressRoute circuit, Microsoft doesn't recommend this solution. To establish connectivity between virtual networks, VNet peering should be implemented instead for the best performance possible. For more information, see [About Virtual Network Peering](../virtual-network/virtual-network-peering-overview.md) and [Manage VNet peering](../virtual-network/virtual-network-manage-peering.md).
+
+## Limitations
+
+Even though ExpressRoute supports virtual network to virtual network connectivity, there are two main limitations with this solution that make it not an ideal choice when compared to VNet peering.
+
+### ExpressRoute virtual network gateway in the data path
+
+Virtual networks that are connected to an ExpressRoute circuit are established by deploying a virtual network gateway. The gateway facilitates the management plane and data path connectivity to virtual machines (VMs) and private endpoints defined in a virtual network. These gateway resources have bandwidth, connections-per-second and packets-per-second limitations. For more information about these limitations, see [About ExpressRoute gateways](expressroute-about-virtual-network-gateways.md). When virtual network to virtual network connectivity goes through ExpressRoute, the virtual network gateway can be the source of bottleneck in terms of bandwidth and data path or control plane limitations. When you configure virtual network peering, the virtual network gateway isn't in the data path. Therefore, you won't experience those limitations seen with VNet to VNet connectivity going through ExpressRoute.
+
+### Higher latency
+
+ExpressRoute connectivity is managed by a pair of Microsoft Enterprise Edge (MSEE) devices located at [ExpressRoute peering locations](expressroute-locations-providers.md#expressroute-locations). ExpressRoute peering locations are physically separate from Azure regions, when virtual network to virtual network connectivity is enabled using ExpressRoute. Traffic from the virtual network leaves the origin Azure region and passes through the MSEE devices at the peering location. Then that traffic will go through Microsoft's global network to reach the destination Azure region. With VNet peering, traffic flows from the origin Azure region directly to the destination Azure region using Microsoft's global network, without the extra hop of the MSEE devices. Since the extra hop is no longer in the data path, you'll see lower latency and an overall better experience with your applications and network traffic.
+
+## Next steps
+
+* Learn more about [Designing for high availability](designing-for-high-availability-with-expressroute.md).
+* Plan for [Disaster recovery](designing-for-disaster-recovery-with-expressroute-privatepeering.md) and [using VPN as a backup](use-s2s-vpn-as-backup-for-expressroute-privatepeering.md).