Merge pull request #276601 from robswain/router-bfd-1339634

ADO 1339634 - Add dual router support

Author: Jill Grant

articles/private-5g-core/collect-required-information-for-a-site.md

@@ -95,11 +95,15 @@ Collect all the values in the following table to define the packet core instance
    | The ID number of the VLAN for the N2 network | S1-MME/N2 VLAN ID |
    | The subnet for the IP address configured for the N2 interface e.g. 10.232.44.0/24 | S1-MME/N2 subnet |
    | The gateway for the IP address configured for the N2 interface e.g. 10.232.44.1. If the subnet does not have a default gateway, use another IP address in the subnet which will respond to ARP requests (such as one of the RAN IP addresses). If there's more than one gNB connected via a switch, choose one of the IP addresses for the gateway. | S1-MME/N2 gateway |
+   | The IP address of one of a redundant pair of access network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down. | S1-MME/N2 gateway BFD endpoint 1 |
+   | The IP address of one of a redundant pair of access network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down.   | S1-MME/N2 gateway BFD endpoint 2 |
    | The virtual network name on port 5 on your Azure Stack Edge Pro device corresponding to the user plane interface on the access network. For 5G, this interface is the N3 interface; for 4G, it's the S1-U interface; for combined 4G and 5G, it's the N3/S1-U interface. </br></br> For an HA deployment, this IP address MUST NOT be in any control plane or user plane subnets; it's used as the destination of routes in the access network gateway routers. | **ASE N3 virtual subnet** (for 5G), **ASE S1-U virtual subnet** (for 4G), or **ASE N3/S1-U virtual subnet** (for combined 4G and 5G). |
    | The ID number of the VLAN for the N3 network | S1-U/N3 VLAN ID |
    | The local IP address for the N3 interface | S1-U/N3 address |
    | The subnet for the IP address configured for the N3 interface e.g. 10.232.44.0/24 | S1-U/N3 subnet |
    | The gateway for the IP address configured for the N3 interface e.g. 10.232.44.1. If the subnet does not have a default gateway, use another IP address in the subnet which will respond to ARP requests (such as one of the RAN IP addresses). If there's more than one gNB connected via a switch, choose one of the IP addresses for the gateway. | S1-U/N3 gateway |
+   | The IP address of one of a redundant pair of access network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down. | S1-U/N3 gateway BFD endpoint 1 |
+   | The IP address of one of a redundant pair of access network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down.   | S1-U/N3 gateway BFD endpoint 2 |
 :::zone-end
 :::zone pivot="ase-pro-2"
    |Value  |Field name in Azure portal  |
@@ -111,10 +115,14 @@ Collect all the values in the following table to define the packet core instance
    | The ID number of the VLAN for the N2 network | S1-MME/N2 VLAN ID |
    | The subnet for the IP address configured for the N2 interface e.g. 10.232.44.0/24 | S1-MME/N2 subnet |
    | The gateway for the IP address configured for the N2 interface e.g. 10.232.44.1. If the subnet does not have a default gateway, use another IP address in the subnet which will respond to ARP requests (such as one of the RAN IP addresses). If there's more than one gNB connected via a switch, choose one of the IP addresses for the gateway. | S1-MME/N2 gateway |
+   | The IP address of one of a redundant pair of access network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down. | S1-MME/N2 gateway BFD endpoint 1 |
+   | The IP address of one of a redundant pair of access network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down.   | S1-MME/N2 gateway BFD endpoint 2 |
    | The ID number of the VLAN for the N3 network | S1-U/N3 VLAN ID |
    | The local IP address for the N3 interface | S1-U/N3 address |
    | The subnet for the IP address configured for the N3 interface e.g. 10.232.44.0/24 | S1-U/N3 subnet |
    | The gateway for the IP address configured for the N3 interface e.g. 10.232.44.1. If the subnet does not have a default gateway, use another IP address in the subnet which will respond to ARP requests (such as one of the RAN IP addresses). If there's more than one gNB connected via a switch, choose one of the IP addresses for the gateway. | S1-U/N3 gateway |
+   | The IP address of one of a redundant pair of access network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down. | S1-U/N3 gateway BFD endpoint 1 |
+   | The IP address of one of a redundant pair of access network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down.   | S1-U/N3 gateway BFD endpoint 2 |
 :::zone-end
 
 ## Collect UE usage tracking values
@@ -146,6 +154,8 @@ For each data network that you want to configure, collect all the values in the
    | The ID number of the VLAN for the data network | SGi/N6 VLAN ID |
    | The IP address of the N6 interface | SGi/N6 address |
    | The gateway for the IP address configured for the N6 interface | SGi/N6 gateway | 
+   | The IP address of one of a redundant pair of data network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down. | SGi/N6 gateway BFD endpoint 1 |
+   | The IP address of one of a redundant pair of data network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down.   | SGi/N6 gateway BFD endpoint 2 |
    | The network address of the subnet from which dynamic IP addresses must be allocated to user equipment (UEs), given in CIDR notation. You don't need this address if you don't want to support dynamic IP address allocation for this site. You identified this in [Allocate user equipment (UE) IP address pools](complete-private-mobile-network-prerequisites.md?pivots=ase-pro-gpu#allocate-user-equipment-ue-ip-address-pools). The following example shows the network address format. </br></br>`192.0.2.0/24` </br></br>Note that the UE subnets aren't related to the access subnet.    |**Dynamic UE IP pool prefixes**|
    | The network address of the subnet from which static IP addresses must be allocated to user equipment (UEs), given in CIDR notation. You don't need this address if you don't want to support static IP address allocation for this site. You identified this in [Allocate user equipment (UE) IP address pools](complete-private-mobile-network-prerequisites.md?pivots=ase-pro-gpu#allocate-user-equipment-ue-ip-address-pools). The following example shows the network address format. </br></br>`203.0.113.0/24` </br></br>Note that the UE subnets aren't related to the access subnet.    |**Static UE IP pool prefixes**|
    | The Domain Name System (DNS) server addresses to be provided to the UEs connected to this data network. You identified this in [Allocate subnets and IP addresses](complete-private-mobile-network-prerequisites.md?pivots=ase-pro-gpu#allocate-subnets-and-ip-addresses). </br></br>This value might be an empty list if you don't want to configure a DNS server for the data network. In this case, UEs in this data network will be unable to resolve domain names. | **DNS Addresses** |
@@ -157,7 +167,9 @@ For each data network that you want to configure, collect all the values in the
    | The name of the data network. This could be an existing data network or a new one you'll create during packet core configuration.                 |**Data network name**|
    | The ID number of the VLAN for the data network | SGi/N6 VLAN ID |
    | The IP address of the N6 interface | SGi/N6 address |
-   | The gateway for the IP address configured for the N6 interface | SGi/N6 gateway | 
+   | The gateway for the IP address configured for the N6 interface | SGi/N6 gateway |
+   | The IP address of one of a redundant pair of data network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down. | SGi/N6 gateway BFD endpoint 1 |
+   | The IP address of one of a redundant pair of data network gateway routers in a highly available (HA) deployment. Used to establish a BFD session between the packet core and each router to maintain service if one router goes down.   | SGi/N6 gateway BFD endpoint 2 |
    | The network address of the subnet from which dynamic IP addresses must be allocated to user equipment (UEs), given in CIDR notation. You don't need this address if you don't want to support dynamic IP address allocation for this site. You identified this in [Allocate user equipment (UE) IP address pools](complete-private-mobile-network-prerequisites.md?pivots=ase-pro-2#allocate-user-equipment-ue-ip-address-pools). The following example shows the network address format. </br></br>`192.0.2.0/24` </br></br>Note that the UE subnets aren't related to the access subnet.    |**Dynamic UE IP pool prefixes**|
    | The network address of the subnet from which static IP addresses must be allocated to user equipment (UEs), given in CIDR notation. You don't need this address if you don't want to support static IP address allocation for this site. You identified this in [Allocate user equipment (UE) IP address pools](complete-private-mobile-network-prerequisites.md?pivots=ase-pro-2#allocate-user-equipment-ue-ip-address-pools). The following example shows the network address format. </br></br>`203.0.113.0/24` </br></br>Note that the UE subnets aren't related to the access subnet.    |**Static UE IP pool prefixes**|
    | The Domain Name System (DNS) server addresses to be provided to the UEs connected to this data network. You identified this in [Allocate subnets and IP addresses](complete-private-mobile-network-prerequisites.md?pivots=ase-pro-2#allocate-subnets-and-ip-addresses). </br></br>This value might be an empty list if you don't want to configure a DNS server for the data network. In this case, UEs in this data network will be unable to resolve domain names. | **DNS Addresses** |

articles/private-5g-core/configure-routers-high-availability.md

@@ -19,8 +19,20 @@ This requires you to deploy a gateway router between the ASE cluster and:
 - the RAN equipment in the access network.
 - the data networks.
 
+The following diagram shows an overview of a highly available deployment with a single access network router and a single data network router.
+
+:::image type="content" source="media/configure-routers-high-availability/highly-available-network.png" alt-text="Diagram showing a highly available deployment with a single access network router and a single data network router.":::
+
 The routers should rapidly detect the failure of an ASE device through a BFD session going down and immediately redirect all traffic to the other ASE. With the recommended settings, BFD should be able to detect failure in about one second, ensuring that traffic should be restored in less than 2.5 seconds. User plane state is replicated across the two ASEs to ensure the backup can take over immediately.
 
+AP5GC only supports a single gateway router IP address per network. Therefore, only a network design where there is either a single gateway router per network or where the gateway routers are deployed in redundant pairs in an active / standby configuration with a floating gateway IP address is supported. The gateway routers in each redundant pair should monitor each other using VRRP (Virtual Router Redundancy Protocol) to provide detection of partner failure.
+
+By default, the packet core creates a single BFD session with the gateway router IP address. For a redundant pair of routers, you must configure the packet core with the interface IP address of each router in addition to the shared gateway address. The packet core will then create an additional BFD session with each router. If one BFD session goes down due to a router failure, service is unaffected because the gateway address is still reachable from the other router. This configuration is added later when you create the site; see [Collect the required information for a site](collect-required-information-for-a-site.md) for details. If you do not configure the interface IP addresses, it is possible for a router to forward traffic to an ASE device that cannot forward it on in the event of a port, link or top-of-rack switch failure.
+
+The following diagram shows the physical layout of the access network with a redundant pair of routers. All links are in a single virtual local area network (VLAN) for the access network. The layout for each data network is similar.
+
+:::image type="content" source="media/configure-routers-high-availability/dual-routers.png" alt-text="Diagram showing the physical layout of the access network with a redundant pair of routers.":::
+
 This how-to guide describes the configuration required on your router or routers to support an HA deployment. The gateway router for the access network and the gateway router for the data networks may be the same device or separate devices.
 
 ## Prerequisites