Merge pull request #291625 from bartpinto/main

Enhance Validate Cables for Nexus Network Fabric Public Documentation

Author: JamesJBarnett

articles/operator-nexus/how-to-validate-cables.md

@@ -11,43 +11,90 @@ ms.date: 04/15/2024
 ---
 # Validate Cables for Nexus Network Fabric
 
-This article explains the  Fabric cable validation, where the primary function of the diagnostic API is to check all fabric devices for potential cabling issues. The Diagnostic API assesses whether the interconnected devices adhere to the Bill of Materials (BOM), classifying them as compliant or noncompliant. The results are presented in a JSON format, encompassing details such as validation status, errors, identifier type, and neighbor device ID. These results are stored in a customer-provided Storage account. It's vital to the overall deployment that errors identified in this report are resolved before moving onto the Cluster deployment step.
+This article explains the  Fabric cable validation, where the primary function of the diagnostic API is to check all fabric devices for potential cabling issues. The Diagnostic API assesses whether the interconnected devices adhere to the Bill of Materials (BOM) and according to the resource Stock Keeping Units (SKUs), classifying them as compliant or noncompliant. Device types include Customer Edge (CE), Top of Rack (TOR), Management (MGMT), and Network Packet Broker (NPB) devices. The results are presented in a JSON format, encompassing details such as validation status, errors, identifier type, and neighbor device ID. These results are stored in a customer-provided Storage account. It's vital to the overall deployment that errors identified in this report are resolved before moving onto the Cluster deployment step.
+
+For BOM details, refer to [Azure Operator Nexus SKUs](./reference-operator-nexus-skus.md)
 
 ## Prerequisites
 
 - Ensure the Nexus Network Fabric is successfully provisioned.
 - Provide the Network Fabric ID and storage URL with WRITE access via a support ticket.
+- The storage account has these prerequisites:
+  - The storage account must be in a different Azure region than the Network Fabric Azure region.
+  - `Storage Blob Data Contributor` role must be assigned to the `Nexus Network Fabric RP` with access assigned to the storage account.
+- Microsoft Support must patch the Nexus Network Fabric with an active storage SAS URL before running cabling validation.
+
+## Creating an Azure Support Request for Patching Nexus Network Fabric:
+  - Refer to [How to create an Azure support request](/azure/azure-portal/supportability/how-to-create-azure-support-request)
+  - Enter `Nexus Network Fabric` for the issue and click `Go`.
+  - Select `Azure Operator Nexus` as the service you're having issue with.
+  - Select `Subscription` where the resource is located.
+  - Select `General` under the resource field and click `Next`.
+  - Select `Network fabric instance provisioning` under the issue prompt and click `Next`.
+  - Now select `Create a support request` at the top of the support blade.
+  - Click `Next` on the `Problem description` tab.
+  - Wait for the `Recommended solution` and then click the link back to 'Return to support request`
+  - Click `Next` on the `Problem description` tab again to continue to `Additional details`
+  - Fill in the required request details to patch Nexus Network Fabric and then click `Next`
+  - Review the information in the `Review + create` tab and then click `Create`.
 
 > [!NOTE]
 > The Storage URL (SAS) is short-lived. By default, it is set to expire in eight hours. If the SAS URL expires, then the fabric must be re-patched.
 
-## Validate cabling
+## Generate the storage URL
+
+To create a container in the storage account, refer to [Create a container](../storage/blobs/blob-containers-portal.md#create-a-container)
+
+> [!NOTE]
+> Enter the name of the container using only lowercase letters.
+
+To generate a SAS URL for the container to provide write access to the Nexus Fabric, refer to [Generate a shared access signature](../storage/blobs/blob-containers-portal.md#generate-a-shared-access-signature)
+
+> [!NOTE]
+> SAS URLs are short lived. By default, it is set to expire in eight hours. If the SAS URL expires, then you must open a Microsoft support ticket to add a new URL.
+
+## Validate Cabling
 
 1. Execute the following Azure CLI command:
 
     ```azurecli
     az networkfabric fabric validate-configuration –resource-group "<NFResourceGroupName>" --resource-name "<NFResourceName>" --validate-action "Cabling" --no-wait --debug  
     ```
 
-    The following (truncated) output appears. Copy the URL through **private preview**. This portion of the URL is used in the following step to check the status of the operation.
+    The following (truncated) output appears. Copy the URL from the `Azure-AsyncOperation` section of the debug output. This portion of the URL is used in the following step to check the status of the operation.
 
     ```azurecli
-    https://management.azure.com/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/providers/Microsoft.ManagedNetworkFabric/locations/EASTUS2EUAP/operationStatuses/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e*A9E6DB3DF5C58D67BD395F7A608C056BC8219C392CC1CE0AD22E4C36D70CEE5C?api-version=2022-01-15-privatepreview&t=638485032018035520&c=MIIHHjCCBgagAwIBAgITfwKWMg6goKCq4WwU2AAEApYyDjANBgkqhkiG9w0BAQsFADBEMRMwEQYKCZImiZPyLGQBGRYDR0JMMRMwEQYKCZImiZPyLGQBGRYDQU1FMRgwFgYDVQQDEw9BTUUgSW5mcmEgQ0EgMDIwHhcNMjQwMTMwMTAzMDI3WhcNMjUwMTI0MTAzMDI3WjBAMT4wPAYDVQQDEzVhc3luY29wZXJhdGlvbnNpZ25pbmdjZXJ0aWZpY2F0ZS5tYW5hZ2VtZW50LmF6dXJlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALMk1pBZQQoNY8tos8XBaEjHjcdWubRHrQk5CqKcX3tpFfukMI0_PVZK-Kr7xkZFQTYp_ItaM2RPRDXx-0W9-mmrUBKvdcQ0rdjcSXDek7GvWS29F5sDHojD1v3e9k2jJa4cVSWwdIguvXmdUa57t1EHxqtDzTL4WmjXitzY8QOIHLMRLyXUNg3Gqfxch40cmQeBoN4rVMlP31LizDfdwRyT1qghK7vgvworA3D9rE00aM0n7TcBH9I0mu-96JE0gSX1FWXctlEcmdwQmXj_U0sZCu11_Yr6Oa34bmUQHGc3hDvO226L1Au-QsLuRWFLbKJ-0wmSV5b3CbU1kweD5LUCAwEAAaOCBAswggQHMCcGCSsGAQQBgjcVCgQaMBgwCgYIKwYBBQUHAwEwCgYIKwYBBQUHAwIwPQYJKwYBBAGCNxUHBDAwLgYmKwYBBAGCNxUIhpDjDYTVtHiE8Ys-
+    cli.azure.cli.core.sdk.policies:     'Azure-AsyncOperation': '<Azure-AsyncOperation-endpoint url>'
+    ```
 
+    The following error indicates the Fabric isn't patched with a valid SAS URL:
+    ```azurecli
+    azure.core.exceptions.HttpResponseError: Operation returned an invalid status 'OK'
     ```
 
 1. You can programmatically check the status of the operation by running the following command:
 
     ```azurecli
     az rest -m get -u "<Azure-AsyncOperation-endpoint url>" 
+    {
+      "endTime": "<OPERATION_COMPLETION_TIME>",
+      "id": "<OPERATION_ID>",
+      "name": "OPERATION_NAME",
+      "properties": {
+        "url": "CABLING_REPORT_STORAGE_URL"
+      },
+      "resourceId": "<FABRIC_RID>",
+      "startTime": "<OPERATION_START_TIME>",
+      "status": "Succeeded"
+    }
     ```
 
     The operation status indicates if the API succeeded or failed.
 
     > [!NOTE]
     > The operation takes roughly 20~40 minutes to complete based on the number of racks.  
 
-1. Download and read the validated results from the storage URL.
+1. Download and read the validated results from the `<CABLING_REPORT_STORAGE_URL>` retuned from the completed command.
 
 Example output is shown in the following sections.
 
@@ -112,7 +159,7 @@ networkFabricInfoSkuId": "M8-A400-A100-C16-ab",
                 },
 ```
 
-### Customer Edge to Top of the Rack switch validation
+### Customer Edge to Top Of Rack (TOR) switch validation
 
 ```azurecli
 { 
@@ -160,36 +207,41 @@ networkFabricInfoSkuId": "M8-A400-A100-C16-ab",
 
 |Status Type  |Definition  |
 |---------|---------|
-|Compliant     | When the status is compliant with the BOM specification         |
-|Non-Compliant     | When the status isn't compliant with the BOM specification         |
-|Unknown     | When the status is unknown         |
+|Compliant     | Validation status is compliant with the BOM specification for the interface. |
+|NonCompliant     | Validation isn't compliant with the BOM specification for the interface.        |
+|Unknown     | Validation is unable to retrieve interface connection details or lldp data isn't returned. The `Unknown` status is returned if the destination device is powered off, missing, disconnected, or unsupported for this interface type. |
 
 #### Validation attributes
 
 |Attribute  |Definition  |
 |---------|---------|
-|`deviceConfiguration`      | Configuration that's available on the device.           |
-|`error`      |  Error from the device        |
+|`deviceConfiguration`      | Configuration that's available on the device. |
+|`error`      |  Error from the device. |
 |`reason`      |  This field is populated when the status of the device is unknown.        |
-|`validationType`      | This parameter indicates what type of validation. (cable & cable specification validations)         |
-|`deviceDestinationResourceId`      |  Azure Resource Manager ID of the connected Neighbor (destination device)        |
-|`roleName`      |  The role of the Network Fabric Device (CE or TOR)        |
+|`validationType`      | This parameter indicates what type of validation. (cable & cable specification validations). |
+|`deviceDestinationResourceId`      |  Azure Resource Manager ID of the connected Neighbor (destination device). |
+|`roleName`      |  The role of the Network Fabric Device (CE or TOR). |
 
-## Known issues and limitations in cable validation
+### Known issues and limitations in cable validation
 
-- Post Validation Connections between TORs and Compute Servers isn't supported.
-- Cable Validation for NPB isn't supported because there's no support for "show lldp neighbors" from Arista.  
+- Cable Validation of connections between TOR and Compute Servers that are powered off or unprovisioned in the Nexus cluster aren't supported. These interfaces show `Unknown` status in the report.
+- Cable Validation of connections between MGMT interfaces and Compute Servers that are powered off or unprovisioned in the Nexus cluster or the Compute Server Controllers aren't supported. These interfaces show `Unknown` status in the report.
+- Cable Validation for NPB isn't supported for `loopback` and `nni-direct` interfaces because there's no vendor support currently for `show lldp neighbors`. These interfaces show `Unknown` status in the report.
 - The Storage URL must be in a different region from the Network Fabric. For instance, if the Fabric is hosted in East US, the storage URL should be outside of East US.
-- Cable validation supports both four rack and eight rack BOMs.
-
-## Generate the storage URL
+- Cable validation supports four racks with 16 Computes per rack and eight racks with 16 Computes per rack BOMs.
+- When destination device is powered off, cables are missing or disconnected, or if validation isn't supported for the interface type, then the interface shows `Unknown` status. **It is important to evaluate all `Unknown` interfaces that are `Not-Connected` against the BOM to determine if repair action is required.**
 
-Refer to [Create a container](../storage/blobs/blob-containers-portal.md#create-a-container) to create a container.  
+### Typical cable validation `NonCompliant` and `Unknown` Issues
 
-> [!NOTE]
-> Enter the name of the container using only lowercase letters.
+|validationType |Status |Error  |Resolution  |
+|---------------|-------|-------|------------|
+| CableValidation |`NonCompliant`|`Device cable connection is incorrect.` | Verify connections on the source and destination interfaces match the BOM. The `deviceConfiguration` can help identify the destination port date returned on the interface. |
+| CableValidation |`Unknown`     |`Unable to fetch data from the device.` | Verify connections on the source and destination interfaces are connected and match the BOM |
+| CableValidation |`NonCompliant`|`Device cabling in <INTERFACE> incorrect.` | The interface isn't connected. Verify connections on the source and destination interfaces are connected  match the BOM. |
+| CableValidation  |`Unknown`     |`Port <INTERFACE> has no connections as per device response.` | Verify connections on the source and destination interfaces are connected and match the BOM. |
+| CableSpecificationValidation  |`Unknown`     |`Unable to fetch Interface Status for <INTERFACE>.` | Verify connections on the source and destination interfaces are connected and match the BOM. |
+| CableSpecificationValidation  |`NonCompliant`     |`Device cable connection is incorrect` | Verify interface card and cables match BOM specification in this interface. |
 
-Refer to [Generate a shared access signature](../storage/blobs/blob-containers-portal.md#generate-a-shared-access-signature) to create the SAS URL of the container. Provide Write permission for SAS.
+## Converting Cable Validation Report to HTML Format
 
-> [!NOTE]
-> ESAS URLs are short lived. By default, it is set to expire in eight hours. If the SAS URL expires, then you must open a Microsoft support ticket to add a new URL.
+Refer to [How to Convert Cable Validation report JSON output to HTML](./howto-convert-cable-validation-html.md).

articles/operator-nexus/howto-convert-cable-validation-html.md

@@ -0,0 +1,149 @@
+---
+title: How to Convert Network Fabric Cable Validation Report to HTML
+description: Learn how to convert Nexus Network Fabric Cable Validation Report to HTML.
+author: bpinto
+ms.author: bpinto
+titleSuffix: Azure Operator Nexus
+ms.service: azure-operator-nexus
+ms.topic: how-to #Required; leave this attribute/value as-is
+ms.date: 12/10/2024
+
+#CustomerIntent: As a < type of user >, I want < what? > so that < why? >.
+---
+# How to Convert Network Fabric Cable Validation Report to HTML
+
+This article explains how to convert a Nexus Network Fabric Cable Validation Report from JSON output to HTML.
+
+## Prerequisites
+
+- Requires Python 3.11 or later
+- Requires Python 3.11 Modules: `json pandas as pd datetime`
+
+## Python Script for Cable Validation JSON to HTML Conversion
+
+```
+import json
+import pandas as pd
+from datetime import datetime
+
+def color_status(val):
+    """
+    Takes a scalar and returns a string with
+    the css property `'color: green'` for compliant,
+    `'color: red'` for noncompliant and black for others
+    """
+    if val == 'Compliant':
+        color = 'green'
+    elif val == 'NonCompliant':
+        color = 'red'
+    else:
+        color = 'black'
+    return 'color: %s' % color
+
+now = datetime.now() # current date and time
+date_time = now.strftime("%m-%d-%Y-%H-%M")
+print("date and time:",date_time)
+
+# Get the file name as input from the user
+file_name = input("Please provide post validation json file: ")
+# Load the JSON data from the file
+with open(file_name, 'r') as f:
+    data = json.load(f)
+
+# Prepare two lists to store the data
+cable_validation_data = []
+cable_specification_validation_data = []
+
+# Loop through each rack in the racks list
+for rack in data['racks']:
+    # Loop through each device in the networkConfiguration list
+    for device in rack['rackInfo']['networkConfiguration']['networkDevices']:
+        # Loop through each interface map for the device
+        for interface_map in device['fixedInterfaceMaps']:
+            # Loop through each validation result for the interface map
+            for validation_result in interface_map['validationResult']:
+                # Append the data to the list based on validation type
+                temp_item = [device['name'], interface_map['name'], validation_result['status'], interface_map['destinationHostname'], interface_map['destinationPort'],validation_result['validationDetails']['deviceConfiguration'], validation_result['validationDetails']['error'] , validation_result['validationDetails']['reason'],'FixedInterface']
+                if validation_result['validationType'] == 'CableValidation':
+                    cable_validation_data.append(temp_item)
+                elif validation_result['validationType'] == 'CableSpecificationValidation':
+                    cable_specification_validation_data.append(temp_item)
+
+        # Check if scaleSpecificInterfaceMaps is not None
+        if device['scaleSpecificInterfaceMaps'] is not None:   
+            # Loop through each scaleSpecificInterface_Map for the interface map
+            for scale_map in device['scaleSpecificInterfaceMaps']:
+                # Loop through each interface map for the device.
+                for interfacemaps in scale_map['InterfaceMaps']:
+                    # Loop through each validation result for the scaleSpecificInterface_Map
+                    for validation_result in interfacemaps['validationResult']:
+                        # Append the data to the list
+                        temp_item = [device['name'], interfacemaps['name'], validation_result['status'], interfacemaps['destinationHostname'], interfacemaps['destinationPort'], validation_result['validationDetails']['deviceConfiguration'], validation_result['validationDetails']['error'] , validation_result['validationDetails']['reason'], 'ScaleSpecificInterface']
+                        if validation_result['validationType'] == 'CableValidation':
+                            cable_validation_data.append(temp_item)
+                        elif validation_result['validationType'] == 'CableSpecificationValidation':
+                            cable_specification_validation_data.append(temp_item)
+
+# Convert the lists to DataFrames
+cable_validation_df = pd.DataFrame(cable_validation_data, columns=['Device Name', 'Interface Map Name', 'Status', 'Destination Hostname', 'Destination Port', 'Device Configuration', 'Error', 'Reason', 'Interface Type'])
+cable_specification_validation_df = pd.DataFrame(cable_specification_validation_data, columns=['Device Name', 'Interface Map Name', 'Status', 'Destination Hostname', 'Destination Port', 'Device Configuration', 'Error', 'Reason', 'Interface Type'])
+
+# Group the DataFrames by 'Status' and append each group's HTML representation to a string
+#html_string = '<html><head><style>table {border-collapse: collapse;} th, td {border: 1px solid black; padding: 5px;}</style></head><body>'
+html_string = """
+<html>
+<head>
+    <style>
+        body {
+            font-family: Arial, sans-serif;
+            max-width: 960px;  /* Set the maximum width of the page */
+            margin: 0 auto;  /* Center the page */
+        }
+        h2 {color: #2A2A2A;}
+        table {border-collapse: collapse; width: 100%;}
+        th, td {border: 1px solid #ddd; padding: 8px;}
+        tr:nth-child(even) {background-color: #f2f2f2;}
+        th {padding-top: 12px; padding-bottom: 12px; text-align: left; background-color: #4CAF50; color: white;}
+    </style>
+</head>
+<body>
+"""
+for status, group_df in cable_validation_df.groupby('Status'):
+    styled_group_df = group_df.style.applymap(color_status, subset=['Status']).set_table_attributes('class="dataframe"')
+    html_string += f'<h2>Cable Validation - {status}</h2>'
+    html_string += styled_group_df.to_html()
+
+for status, group_df in cable_specification_validation_df.groupby('Status'):
+    styled_group_df = group_df.style.applymap(color_status, subset=['Status']).set_table_attributes('class="dataframe"')
+    html_string += f'<h2>Cable Specification Validation - {status}</h2>'
+    html_string += styled_group_df.to_html()
+
+html_string += '</body></html>'
+
+# Write the string to an HTML file
+with open('CableValidationAndSpecification-{filename}.html'.format(filename = date_time), 'w') as f:
+    f.write(html_string)
+```
+
+## Usage
+To run the conversion tool, execute the following command:
+```
+python cable-html.py
+..
+Please provide post validation json file: <CABLE_VALIDATION_FILENAME>.json
+```
+
+The report output has filename `CableValidationAndSpecification-<DATE>.html`.
+
+## Cable Validation Report HTML results
+
+The report is separated into the following sections:
+- Cable Validation - Compliant
+- Cable Validation - NonCompliant
+- Cable Validation - Unknown
+- Cable Specification Validation - Compliant
+- Cable Specification Validation - NonCompliant
+- Cable Specification Validation - Unknown
+
+Sample report:
+:::image type="content" source="media\cable-validation-html.png" alt-text="Screenshot that shows the sample cable validation report." lightbox="media\cable-validation-html.png":::