Merge pull request #297434 from sushantjrao/break-glass-setup

Release 8.1

Author: PMEds28

articles/operator-nexus/TOC.yml

@@ -230,6 +230,12 @@
           href: howto-upgrade-os-of-terminal-server.md
         - name: How to restrict serial port access and set timeout on terminal-server
           href: howto-restrict-serial-port-access-and-set-timeout-on-terminal-server.md
+        - name: How to configure BGP prefix limit on Customer Edge (CE) devices for Azure Operator Nexus
+          href: howto-configure-bgp-prefix-limit-on-customer-edge-devices.md
+        - name: BMP log streaming in Azure Operator Nexus Network Fabric 
+          href: concepts-bmp-log-streaming.md
+        - name: How to enable / disable BMP log streaming Azure Operator Nexus
+          href: howto-enable-log-streaming.md
     - name: Cluster
       expanded: false
       items:

articles/operator-nexus/concepts-bmp-log-streaming.md

@@ -0,0 +1,89 @@
+---
+title: BMP log streaming in Azure Operator Nexus Network Fabric 
+description: An overview of BGP Monitoring Protocol (BMP), its importance, and key components for effective log monitoring.
+author: sushantjrao
+ms.author: sushrao
+ms.service: azure-operator-nexus
+ms.topic: conceptual
+ms.date: 04/01/2025
+ms.custom: template-concept
+---
+
+# Introduction to BMP
+
+The **BGP Monitoring Protocol (BMP)** is a protocol designed to monitor BGP sessions. It provides a standardized method for collecting information about BGP sessions, which can be used for analysis, troubleshooting, and ensuring the stability and security of the network.
+
+The BGP Monitoring Protocol (BMP) allows a monitoring station to connect to a router and collect all the BGP announcements received from the router's BGP peers. The announcements are sent to the station in the form of **BMP Route Monitoring messages** formed from path information in the router's **BGP Adj-Rib-In tables**. A BMP speaker may choose to send either **pre-policy routes, post-policy routes, or both**.
+
+BMP is **unidirectional**: BMP sends messages only from the router to the monitoring station, never the other way around. The router's configuration controls the information sent. Besides Route Monitoring messages, BMP also sends these messages:
+
+- **Initiation**: Sent at the beginning of a session and used to identify the router.
+- **Termination**: Optionally sent at the end of a session to indicate why the session is being closed.
+- **Peer Up**: Used to indicate if a BGP peer is in **Established** state.
+- **Peer Down**: Used to indicate that a BGP peer has gone out of **Established** state.
+
+Connections between the router and BMP stations use **TCP**. The router can either passively listen for incoming connections from a station or actively initiate them, configurable per station. Only **one connection per BMP station** is allowed at a time. If a station reconnects, the router closes the old session and starts a new BMP session with the new connection.
+
+## Importance of BMP log monitoring
+
+BMP log monitoring is crucial for maintaining the **health and performance** of BGP sessions. By continuously monitoring BMP logs, network administrators can detect anomalies, identify potential issues, and take proactive measures to prevent network disruptions. Effective BMP log monitoring helps in:
+
+- Ensuring the **stability and reliability** of BGP sessions.
+- Detecting and mitigating **security threats**.
+- Analyzing **network performance** and identifying optimization opportunities.
+- Troubleshooting and resolving **BGP-related issues** promptly.
+
+## Key Components of BMP log monitoring
+
+### **1. Data collection**
+BMP logs provide detailed information about BGP sessions, including **route updates, peer status, and error messages**. Collecting this data is the first step in effective BMP log monitoring.
+
+### **2. Data storage**
+Storing BMP logs in a **centralized and secure location** is essential for easy access and analysis. This can be achieved using **log management systems** or **databases**.
+
+### **3. Data analysis**
+Analyzing BMP logs helps in identifying **patterns, trends, and anomalies**. Advanced analytics tools and techniques, such as **machine learning**, can be used to gain deeper insights into the network's behavior.
+
+### **4. Alerting and reporting**
+Setting up **alerts** for specific events or thresholds in BMP logs ensures that network administrators are **promptly notified** of any issues. **Regular reports** provide a comprehensive overview of the network's health and performance.
+
+## Restrictions and limitations of BMP log streaming
+
+### Nexus Network Fabric (NF) restrictions
+
+Nexus NF shall not enable BMP Log streaming for the following BGP neighbors:
+
+- INTERCEGLOBAL
+- INTERCEEVPN
+- INTERCEOPTION-A
+- CETORUNDERLAY
+- EVPN_OVERLAY
+
+Nexus NF shall not support BMP Log streaming for RT-Membership and EVPN Address-family.
+
+Nexus NF shall not support monitoring address-families at the station level, even though Nexus NF provides ARM API at the station level by facilitating the address-family. This limitation exists because Arista supports BGP Global level rather than BMP Station level.
+
+### L3ISD requirements
+
+- L3ISD must be enabled prior to associating the L3ISDs as Monitored Networks of any Network Monitors.
+- L3ISD must be enabled prior to associating the L3ISDs Internal/External Network as Station Network of any Network Monitors.
+- L3ISD shall not be disabled if the respective L3ISD internal/external network is associated under Station Network of any Network Monitors.
+
+### Unsupported features
+
+Nexus shall not support the following features:
+
+- BMP support for VRF Filtering.
+- BMP support for local rib VPN imported routes.
+- BMP support for exporting Adj-Rib-Out.
+
+### Peer-Address monitoring
+
+Nexus NF shall not support excluding the monitoring of peer-address of neighbor groups where the neighbor group is configured with BGP listen range. This limitation is due to Arista's inability to support excluding the monitoring of certain addresses of neighbors configured with BGP listen-range.
+
+### Network monitors
+
+Nexus shall support a maximum of four Network Monitors (BMP Stations).
+
+## Next steps
+[How to enable / disable BMP log streaming](./howto-enable-log-streaming.md)

articles/operator-nexus/howto-append-custom-suffix-to-interface-descriptions.md

@@ -11,7 +11,7 @@ ms.custom: template-how-to
 
 # Append custom suffix to interface descriptions in Azure Operator Nexus Network Fabric
 
-TThis guide explains how to append a user-defined suffix (`additionalDescription`) to interface descriptions in Azure Operator Nexus Network Fabric. The primary interface description is `read-only,` but the `additionalDescription` property provides enhanced flexibility for operational annotations. This approach allows users to customize interface descriptions for specific maintenance or operational requirements without altering the system-generated description.
+This guide explains how to append a user-defined suffix (`additionalDescription`) to interface descriptions in Azure Operator Nexus Network Fabric. The primary interface description is `read-only,` but the `additionalDescription` property provides enhanced flexibility for operational annotations. This approach allows users to customize interface descriptions for specific maintenance or operational requirements without altering the system-generated description.
 
 ## Prerequisites
 
@@ -31,7 +31,7 @@ az networkfabric interface show -g "example-rg" \
   --resource-name "example-interface" --query description
 ```
 
-### Parameter Details  
+#### Parameter details  
 
 | Parameter                     | Short Form | Description |
 |--------------------------------|-----------|-------------|
@@ -54,7 +54,7 @@ az networkfabric interface update --additional-description "example-description"
   --resource-name "example-interface"
 ```
 
-### Parameter Details  
+#### Parameter details  
 
 | Parameter                | Description                                      | Constraints |
 |--------------------------|--------------------------------------------------|-------------|
@@ -70,7 +70,7 @@ After updating the description, apply the changes to the Fabric:
 ```Azure CLI
 az networkfabric fabric commit-configuration --resource-group "example-rg" --resource-name "example-fabric"
 ```
-Parameter Details:
+#### Parameter details
 
 | Parameter            | Short Form | Description |
 |----------------------|-----------|-------------|
@@ -101,7 +101,7 @@ az networkfabric interface update --additional-description "example-description"
   --resource-name "example-interface"
 ```
 
-### Parameter Details  
+#### Parameter details  
 
 | Parameter                | Description                                      | Constraints |
 |--------------------------|--------------------------------------------------|-------------|
@@ -118,7 +118,7 @@ After removing the suffix, apply the changes to the Fabric:
 az networkfabric fabric commit-configuration --resource-group "example-rg" --resource-name "example-fabric"
 ```
 
-Parameter Details:
+#### Parameter details
 
 | Parameter            | Short Form | Description |
 |----------------------|-----------|-------------|
@@ -131,9 +131,33 @@ Once committed, the interface description reverts to its original state:
 AR-CE2(Fab3-AR-CE2):Et1/1 to CR1-TOR1(Fab3-CP1-TOR1)-Port23
 ```
 
+## Network interface updates
+
+Updates have been made to the network interface of the network device to standardize the interface description. Additionally, these updates now link the interface to the ARM resource ID of the connected interface for better management and tracking.
+
+### Standardized interface descriptions
+
+Interface descriptions follow a consistent format:
+
+Source Device to Destination Device (including hostname and interface name).
+
+### Example
+AR-CE2 (Fab3-AR-CE2): Et1/1 to CR1-TOR1 (Fab3-CP1-TOR1) - Port23
+
+### connectedTo property
+
+The `connectedTo` property returns the ARM resource ID of the connected interface, where available.
+
+### Comparison of old and new values
+
+| Example | Previous Value | New Value |
+|---------|---------------|-----------|
+| **Example 1** | CR1-TOR1-Port23 | `/subscriptions/<subscriptionID>/resourceGroups/<resourceGroupName>/providers/Microsoft.ManagedNetworkFabric/networkDevices/fab3nf-CompRack1-TOR1/networkInterfaces/Ethernet23-1` |
+| **Example 2** | AR-CE2 (Fab3-AR-CE2): Et1/1 to CR1-TOR1 (Fab3-CP1-TOR1) - Port23 | `/subscriptions/<subscriptionID>/resourceGroups/<resourceGroupName>/providers/Microsoft.ManagedNetworkFabric/networkDevices/fab3nf-CompRack1-TOR1/networkInterfaces/Ethernet23-1` |
+
 ## Supported interface types
 
-This feature is available for the following interface types:
+All the above features are available for the following interface types:
 
 - **Agg Rack CE**  
 - **Agg Rack Management**  
@@ -142,4 +166,4 @@ This feature is available for the following interface types:
 - **NPB Device**  
 
 > [!Note]  
-> **Existing deployments** will retain their **current descriptions** until Fabric instances are **migrated to Release 8.0**. After migration, users must update descriptions via the **API**.
+> For non-NF managed devices, such as PE \ Storage Device, the `connectedTo` property will continue to reflect value as a `string` with no active link.

articles/operator-nexus/howto-configure-bgp-prefix-limit-on-customer-edge-devices.md

@@ -0,0 +1,196 @@
+---
+title: How to configure BGP prefix limit on Customer Edge (CE) devices for Azure Operator Nexus
+description: Learn the process for configuring BGP prefix limit on Customer Edge (CE) devices for Azure Operator Nexus
+author: sushantjrao 
+ms.author: sushrao
+ms.date: 04/02/2025
+ms.topic: how-to
+ms.service: azure-operator-nexus
+ms.custom: template-how-to, devx-track-azurecli
+---
+
+# BGP prefix limiting overview
+
+BGP (Border Gateway Protocol) prefix limiting is an essential overload protection mechanism for Customer Edge (CE) devices. It helps prevent the Nexus fabric from being overwhelmed when a Nexus tenant advertises an excessive number of BGP routes into a Nexus Virtual Routing and Forwarding (VRF) instance. This feature ensures network stability and security by controlling the number of prefixes received from BGP peers.
+
+## Configuration of BGP prefix limits
+
+BGP prefix limits can be configured using two primary parameters:
+
+- **max-routes (hard limits)**: This parameter sets the maximum number of prefixes a BGP router accepts from a neighbor. If the limit is exceeded, the BGP session with that neighbor is terminated to prevent overloading the router.
+  
+- **warn-threshold (soft limits)**: The warn-threshold parameter sets a warning threshold below the max-routes limit. When the number of prefixes received from a neighbor exceeds this threshold, a warning is generated, but the BGP session isn't terminated. This policy allows network administrators to take corrective action before the hard limit is reached.
+
+### Hard limits (max-routes)
+
+The `max-routes` parameter specifies the maximum number of prefixes that a BGP router can accept from a neighbor. If the number exceeds this limit, the BGP session with that neighbor is terminated. This threshold is a "hard" limit to protect the router from excessive load and to maintain network stability.
+
+### Soft limits (warn-threshold)
+
+The `warn-threshold` parameter is a "soft" limit. When the number of prefixes exceeds this threshold, a warning is triggered, but the BGP session remains active. This safeguard serves as a precautionary measure, allowing administrators to intervene before reaching the hard limit.
+
+To configure **BGP Prefix Limit** on **Customer Edge (CE)** devices for **Azure Operator Nexus**, follow the steps below. This configuration includes setting the prefix limits for BGP sessions to manage network stability and prevent the Nexus fabric from being overwhelmed when a tenant advertises excessive BGP routes.
+
+
+### Prerequisites
+
+- Ensure that the **Network Fabric (NF)** is upgraded to the supported version or later.
+
+- Verify that your **Customer Edge (CE)** devices are running on compatible software.
+
+- Check that the **peer groups** for both **IPv4** and **IPv6** address-families are properly set up for internal networks.
+
+### Steps to configure BGP prefix limits
+
+#### Step 1: Define BGP prefix limits
+
+You need to configure the BGP prefix limits using the parameters `maximumRoutes` and `threshold`.
+
+- **`maximumRoutes`**: This parameter defines the maximum number of BGP prefixes the router accepts from a BGP peer.
+
+- **`threshold`**: This parameter defines the warning threshold as a percentage of the `maximumRoutes`. When the number of prefixes exceeds this threshold, a warning is generated.
+
+#### Step 2: Configure on the CE device
+
+##### Example 1: BGP Prefix Limit with automatic restart
+
+This configuration will automatically restart the session after a defined idle time when the prefix limit is exceeded.
+
+```json
+{
+  "prefixLimits": {
+    "maximumRoutes": 5000,
+    "threshold": 80,
+    "idleTimeExpiry": 100
+  }
+}
+```
+
+- **Explanation**:
+
+  - **maximumRoutes**: 5,000 routes are the limit for the BGP session.
+
+  - **threshold**: A warning is triggered when the prefix count reaches 80% (4,000 routes).
+
+  - **idleTimeExpiry**: If the session is shut down, it will restart automatically after 100 seconds of idle time.
+
+##### Example 2: BGP prefix limit without automatic restart
+
+This configuration shuts down the session when the maximum prefix limit is reached, but manual intervention is required to restart the session.
+
+```json
+{
+  "prefixLimits": {
+    "maximumRoutes": 5000,
+    "threshold": 80
+  }
+}
+```
+
+- **Explanation**:
+
+  - **maximumRoutes**: 5,000 routes are the limit for the BGP session.
+
+  - **threshold**: A warning is triggered when the prefix count reaches 80% (4,000 routes).
+
+  - No automatic restart; manual intervention is required to restart the session.
+
+##### Example 3: Hard-Limit drop BGP sessions
+
+This configuration drops extra routes if the prefix limit is exceeded without maintaining a cache of the dropped routes.
+
+```json
+{
+  "prefixLimits": {
+    "maximumRoutes": 5000
+  }
+}
+```
+
+- **Explanation**:
+
+  - **maximumRoutes**: 5,000 routes are the limit for the BGP session.
+
+  - Once the limit is reached, the CE device drops any extra prefixes received from the BGP peer.
+
+##### Example 4: Hard-Limit warning only
+
+This configuration generates a warning once the prefix count reaches a certain percentage of the maximum limit but does not shut down the session.
+
+```json
+{
+  "prefixLimits": {
+    "maximumRoutes": 8000,
+    "threshold": 75,
+    "warning-only": true
+  }
+}
+```
+
+- **Explanation**:
+
+  - **maximumRoutes**: 8,000 routes are the limit for the BGP session.
+
+  - **threshold**: A warning is generated when the prefix count reaches 75% (6,000 routes).
+
+  - The session isn't shut down. This configuration is used to only generate a warning without taking any session-terminating action.
+
+#### Step 3: Apply Configuration Using Azure CLI
+
+You can use Azure CLI commands to apply the BGP prefix limits to the external network configuration for Nexus.
+
+- **With Automatic Restart**:
+
+   ```bash
+   az networkfabric externalnetwork create --resource-group <resource-group> --fabric-name <fabric-name> --network-name <network-name> --prefix-limits '{"maximumRoutes": 5000, "threshold": 80, "idleTimeExpiry": 100}'
+   ```
+
+- **Without Automatic Restart**:
+
+   ```bash
+   az networkfabric externalnetwork create --resource-group <resource-group> --fabric-name <fabric-name> --network-name <network-name> --prefix-limits '{"maximumRoutes": 5000, "threshold": 80}'
+   ```
+
+- **Hard-Limit Drop BGP Sessions**:
+
+   ```bash
+   az networkfabric externalnetwork create --resource-group <resource-group> --fabric-name <fabric-name> --network-name <network-name> --prefix-limits '{"maximumRoutes": 5000}'
+   ```
+
+- **Hard-Limit Warning Only**:
+
+   ```bash
+   az networkfabric externalnetwork create --resource-group <resource-group> --fabric-name <fabric-name> --network-name <network-name> --prefix-limits '{"maximumRoutes": 8000, "threshold": 75, "warning-only": true}'
+   ```
+
+#### Step 4: Monitor and validate the configuration
+
+After applying the configuration, ensure to monitor the **BGP session** and validate whether the prefix limits are being enforced properly. You can check the status of the BGP session by using the following command:
+
+```bash
+show ip bgp summary
+```
+
+Look for the session states and the number of prefixes advertised by each peer. If the limits are being hit, you should see the session state change to **Established** or **Idle** based on the configuration.
+
+### Considerations
+
+- **Threshold and Maximum Limits**: Ensure that you set appropriate thresholds to avoid unnecessary session terminations while still protecting the network from overload.
+
+- **Automatic vs. Manual Restart**: Depending on your network operations, choose between automatic and manual restart options. Automatic restart is useful for minimizing manual intervention, but manual restart may give network administrators more control over recovery.
+
+## Handling BGP Prefix Limits for Different Networks
+
+### Internal network
+
+The platform supports Layer 3 Isolation Domain (L3IsolationDomain) for tenant workloads. It performs device programming on Nexus instances and Arista devices with peer groups for both IPv4 and IPv6 address families.
+
+### External network Option B (PE)
+
+For external network configuration, only the **hard-limit warning-only** option is supported. Nexus supports this configuration via the ARM API under the **NNI optionBlayer3Configuration** with the `maximumRoutes` parameter.
+
+### NNI Option A
+
+For NNI Option A, only a single peer group is allowed. IPv4 over IPv6 and vice versa aren't supported. Warning-only mode is available for handling prefix limits.
+
+By following this guide, you can configure BGP prefix limits effectively to protect your network from overload and ensure that BGP sessions are properly managed for both internal and external networks.