Sync release-hotfixes with main

Sync release-hotfixes with main

Author: sethmanheim

AKS-Arc/TOC.yml

@@ -181,6 +181,8 @@
       href: check-vm-sku.md
     - name: Connectivity issues with MetalLB
       href: load-balancer-issues.md
+    - name: Troubleshoot general network validation errors
+      href: network-validation-errors.md
     - name: Network validation error due to .local domain
       href: network-validation-error-local.md
   - name: Reference

AKS-Arc/network-validation-errors.md

@@ -0,0 +1,102 @@
+---
+title: Troubleshoot network validation errors
+description: Learn how to troubleshoot general network validation errors in AKS Arc.
+author: sethmanheim
+ms.author: sethm
+ms.topic: troubleshooting
+ms.date: 05/07/2025
+ms.reviewer: pradwivedi
+ms.lastreviewed: 05/06/2025
+
+---
+
+# Troubleshoot network validation errors
+
+This article describes how to identify and resolve various network validation errors you might encounter during cluster creation. The article emphasizes the importance of pre-checks for early issue detection. These errors are detected by pre-checks designed to highlight issues early, allowing for easier resolution before the cluster is created.
+
+The article summarizes error codes, their potential causes, and actionable mitigation steps to help you resolve issues effectively.
+
+## CloudAgentConnectivityError
+
+Error: Network validation failed during cluster creation.
+
+### Description
+
+Detailed message: `Not able to connect to http://cloudagent.contoso.local:50000. Error returned: action failed after 5 attempts: Get "http://cloudagent.contoso.local:50000": dial tcp: lookup http://cloudagent.contoso.local: Temporary failure in name resolution`
+
+The MOC cloud agent is created using one of the IP addresses from the [Management IP pool](/azure/azure-local/plan/cloud-deployment-network-considerations#management-ip-pool) on port 5500 and the control plane node VM is given IP addresses from the Arc VM logical network. This error occurs when the MOC cloud agent is not reachable from the control plane VM, or when the DNS servers specified in the Arc VM logical network are unable to resolve the MOC cloud agent FQDN.
+
+### Causes of failure
+
+Logical network IP addresses can't connect to management IP pool addresses, due to:
+
+- Incorrect DNS server resolution.
+- Firewall rules between the Arc VM logical network and the cloud agent endpoint.
+- The logical network is in a different VLAN than the management IP pool and there's no cross-VLAN connectivity.
+
+### Mitigation
+
+To resolve this error, you can take the following steps:
+
+- Make sure that the DNS servers specified in the Arc VM logical network can resolve the MOC cloud agent FQDN.
+- Make sure that the logical network IP addresses can connect to all the management IP pool addresses on the required ports. For a detailed list of ports that need to be opened, see [AKS network port and cross-VLAN requirements](aks-hci-network-system-requirements.md#network-port-and-cross-vlan-requirements).
+
+## InternetConnectivityError
+
+Error: Network validation failed during cluster creation.
+
+### Description
+
+Detailed message: `Not able to connect to https://mcr.microsoft.com. Error returned: action failed after 5 attempts: Get "https://mcr.microsoft.com": dial tcp: lookup mcr.microsoft.com on <>: read udp <>: i/o timeout`.
+
+This error indicates that the required URLs are not reachable from the AKS cluster control plane node VM.
+
+### Causes of failure
+
+- Control plane node VM has no outbound internet access.
+- Required URLs aren't allowed through the firewall.
+
+### Mitigation
+
+To resolve this error, ensure that the logical network IP addresses have outbound internet access. If there's a firewall, ensure that the [AKS required URLs](aks-hci-network-system-requirements.md#firewall-url-exceptions) are accessible from the Arc VM logical network.
+
+## VMNotReachableError
+
+Error: Network validation failed during cluster creation.
+
+### Description
+
+Detailed message: `VM IP : <> is not reachable from management cluster`.
+
+This error indicates that the AKS cluster control plane VM is not reachable from the Arc Resource Bridge (ARB).
+
+### Causes of failure
+
+The Arc VM logical network is not reachable from management IP pool addresses.
+
+### Mitigation
+
+To resolve this error, you can take the following steps:
+
+- Make sure that the management IP pool addresses can reach the logical network IP addresses.
+- For a detailed list of ports that need to be opened, see [AKS network port and cross-VLAN requirements](aks-hci-network-system-requirements.md#network-port-and-cross-vlan-requirements).
+
+## DNSResolutionError
+
+This error occurs when DNS servers specified in the Arc VM logical network can't resolve the MOC cloud FQDN or the required URLs.
+
+### Causes of failure
+
+DNS servers specified in a logical network can't resolve the MOC cloud FQDN or the required URLs.
+
+### Mitigation
+
+To resolve this error, check the DNS servers specified in the logical network so that they can resolve the MOC cloud FQDN or the required URLs.
+
+## Contact Microsoft Support
+
+If problems persist, [collect AKS cluster logs](get-on-demand-logs.md) before you [create a support request](aks-troubleshoot.md#open-a-support-request).
+
+## Next steps
+
+[Troubleshoot issues in AKS enabled by Azure Arc](aks-troubleshoot.md)

azure-local/concepts/firewall-requirements.md

@@ -65,6 +65,11 @@ For a consolidated list of endpoints for Japan East that includes Azure Local, A
 For a consolidated list of endpoints for South Central US that includes Azure Local, Arc-enabled servers, ARB, and AKS, use:
 - [Required endpoints in South Central US for Azure Local](https://github.com/Azure/AzureStack-Tools/blob/master/HCI/SouthCentralUSEndpoints/southcentralus-hci-endpoints.md)
 
+## Required firewall URLs for Azure Local in Azure Government regions
+
+For a consolidated list of endpoints for US Gov Virginia that includes Azure Local, Arc-enabled servers, ARB, and AKS, use:
+- [Required endpoints in US Gov Virginia for Azure Local](https://github.com/CristianEdwards/AzureStack-Tools/blob/master/HCI/usgovvirginia-hci-endpoints/usgovvirginia-hci-endpoints.md)
+
 ## Firewall requirements for OEMs
 
 Depending on the OEM you are using for Azure Local you may need to open additional endpoints in your firewall.

azure-local/concepts/software-defined-networking.md

@@ -5,7 +5,7 @@ author: AnirbanPaul
 ms.author: anpaul
 ms.topic: conceptual
 ms.service: azure-local
-ms.date: 04/17/2023
+ms.date: 05/06/2025
 ---
 
 # Software Defined Networking (SDN) in Azure Stack HCI and Windows Server
@@ -21,7 +21,7 @@ Virtual network elements such as [Hyper-V Virtual Switch](/windows-server/virtua
 There are three major SDN components, and you can choose which you want to deploy: Network Controller, Software Load Balancer, and Gateway.
 
    > [!NOTE]
-   > SDN is not supported on stretched (multi-site) clusters.
+   > SDN isn't supported on stretched (multi-site) clusters.
 
 ## Network Controller
 
@@ -45,7 +45,7 @@ You have the option to [deploy SDN Network Controller using SDN Express](../mana
 Gateways are used for routing network traffic between a virtual network and another network, either local or remote. Gateways can be used to:
 
 - Create secure site-to-site IPsec connections between SDN virtual networks and external customer networks over the internet.
-- Create Generic Routing Encapsulation (GRE) connections between SDN virtual networks and external networks. The difference between site-to-site connections and GRE connections is that the latter is not an encrypted connection. For more information about GRE connectivity scenarios, see [GRE Tunneling in Windows Server](/windows-server/remote/remote-access/ras-gateway/gre-tunneling-windows-server).
+- Create Generic Routing Encapsulation (GRE) connections between SDN virtual networks and external networks. The difference between site-to-site connections and GRE connections is that the latter isn't an encrypted connection. For more information about GRE connectivity scenarios, see [GRE Tunneling in Windows Server](/windows-server/remote/remote-access/ras-gateway/gre-tunneling-windows-server).
 - Create Layer 3 connections between SDN virtual networks and external networks. In this case, the SDN gateway simply acts as a router between your virtual network and the external network.
 
 Gateways use [Border Gateway Protocol](/windows-server/remote/remote-access/bgp/border-gateway-protocol-bgp) to advertise GRE endpoints and establish point-to-point connections. SDN deployment creates a default gateway pool that supports all connection types. Within this pool, you can specify how many gateways are reserved on standby in case an active gateway fails.

azure-local/concepts/system-requirements.md

@@ -6,7 +6,7 @@ ms.author: alkohli
 ms.topic: how-to
 ms.service: azure-local
 ms.custom: references_regions
-ms.date: 04/29/2025
+ms.date: 05/06/2025
 ---
 
 # System requirements for Azure Stack HCI, version 22H2
@@ -112,7 +112,7 @@ For best results, adhere to the following requirements:
 An Azure Stack HCI cluster requires a reliable high-bandwidth, low-latency network connection between each server node.
 
 - Verify at least one network adapter is available and dedicated for cluster management.
-- Verify that physical switches in your network are configured to allow traffic on any VLANs you'll use.
+- Verify that physical switches in your network are configured to allow traffic on any VLANs you use.
 
 For physical networking considerations and requirements, see [Physical network requirements](physical-network-requirements.md).
 
@@ -153,7 +153,7 @@ If you use Windows Admin Center to [create](../deploy/create-cluster.md) or [man
 
 - If you're running Windows Admin Center on a server (instead of a local PC), use an account that's a member of the Gateway Administrators group, or the local Administrators group on the Windows Admin Center server.
 
-- Verify that your Windows Admin Center management computer is joined to the same Active Directory domain in which you'll create the cluster, or joined to a fully trusted domain. The servers that you'll cluster don't need to belong to the domain yet; they can be added to the domain during cluster creation.
+- Verify that your Windows Admin Center management computer is joined to the same Active Directory domain in which you create the cluster, or joined to a fully trusted domain. The servers that you'll cluster don't need to belong to the domain yet; they can be added to the domain during cluster creation.
 
 ## Maximum supported hardware specifications
 

azure-local/deploy/deployment-arc-register-server-permissions.md

@@ -3,7 +3,7 @@ title: Register your Azure Local machines with Azure Arc and assign permissions
 description: Learn how to Register your Azure Local machines with Azure Arc and assign permissions for deployment. 
 author: alkohli
 ms.topic: how-to
-ms.date: 05/02/2025
+ms.date: 05/06/2025
 ms.author: alkohli
 ms.service: azure-local
 ms.custom: devx-track-azurepowershell
@@ -129,7 +129,7 @@ Before you begin, make sure you complete the following prerequisites:
     # [Output](#tab/output)
 
     Here's a sample output of a successful registration of your machines:
-    
+
     ```output
     PS C:\Users\Administrator> Invoke-AzStackHciArcInitialization -SubscriptionID $Subscription -ResourceGroup $RG -TenantID $Tenant -Region $Region -Cloud "AzureCloud" -ArmAccessToken $ARMtoken -AccountID $id
     >>
@@ -154,17 +154,13 @@ Before you begin, make sure you complete the following prerequisites:
 
 4. After the script completes successfully on all the machines, verify that:
 
-
     1. Your machines are registered with Arc. Go to the Azure portal and then go to the resource group associated with the registration. The machines appear within the specified resource group as **Machine - Azure Arc** type resources.
 
         :::image type="content" source="media/deployment-arc-register-server-permissions/arc-servers-registered-1.png" alt-text="Screenshot of the Azure Local machines in the resource group after the successful registration." lightbox="./media/deployment-arc-register-server-permissions/arc-servers-registered-1.png":::
 
-
-
 > [!NOTE]
 > Once an Azure Local machine is registered with Azure Arc, the only way to undo the registration is to install the operating system again on the machine.
 
-
 ## Assign required permissions for deployment
 
 This section describes how to assign Azure permissions for deployment from the Azure portal.
@@ -188,17 +184,9 @@ This section describes how to assign Azure permissions for deployment from the A
     - **Key Vault Secrets Officer**: This permission is required to read and write secrets in the key vault used for deployment.
     - **Key Vault Contributor**: This permission is required to create the key vault used for deployment.
     - **Storage Account Contributor**: This permission is required to create the storage account used for deployment.
- 
 
 1. In the right pane, go to **Role assignments**. Verify that the deployment user has all the configured roles.
 
-1. In the Azure portal, go to **Microsoft Entra Roles and Administrators** and assign the **Cloud Application Administrator** role permission at the Microsoft Entra tenant level.
-
-    :::image type="content" source="media/deployment-arc-register-server-permissions/cloud-application-administrator-role-at-tenant.png" alt-text="Screenshot of the Cloud Application Administrator permission at the tenant level." lightbox="./media/deployment-arc-register-server-permissions/cloud-application-administrator-role-at-tenant.png":::
-
-    > [!NOTE]
-    > The Cloud Application Administrator permission is temporarily needed to create the service principal. After deployment, this permission can be removed.
-
 ## Next steps
 
 After setting up the first machine in your instance, you're ready to deploy using Azure portal:

azure-local/deploy/deployment-azure-resource-manager-template.md

@@ -3,7 +3,7 @@ title: Azure Resource Manager template deployment for Azure Local, version 23H2
 description: Learn how to prepare and then deploy Azure Local instance, version 23H2 using the Azure Resource Manager template.
 author: alkohli
 ms.topic: how-to
-ms.date: 05/01/2025
+ms.date: 05/06/2025
 ms.author: alkohli
 ms.reviewer: alkohli
 ms.service: azure-local
@@ -58,7 +58,7 @@ A Resource Manager template creates and assigns all the resource permissions req
 With all the prerequisite and preparation steps complete, you're ready to deploy using a known good and tested Resource Manager deployment template and corresponding parameters JSON file. Use the parameters contained in the JSON file to fill out all values, including the values generated previously.
 
 > [!IMPORTANT]
-> In this release, make sure that all the parameters contained in the JSON value are filled out including the ones that have a null value. If there are null values, then those need to be populated or the validation fails.
+> In this release, make sure that all the parameters contained in the JSON value are filled out including the ones that have a null value. If there are null values, then those parameters need to be populated or the validation fails.
 
 1. In the Azure portal, go to **Home** and select **+ Create a resource**.
 

azure-local/deploy/media/deployment-arc-register-server-permissions/cloud-application-administrator-role-at-tenant.png

@@ -5,7 +5,7 @@ author: alkohli
 ms.author: alkohli
 ms.topic: how-to
 ms.service: azure-local
-ms.date: 04/29/2025
+ms.date: 05/06/2025
 ---
 
 # Failover cluster maintenance procedures
@@ -40,7 +40,7 @@ Before either shutting down or restarting a server, you should pause the server
 
    *Pause server(s) for maintenance: Are you sure you want to pause server(s)? This moves workloads, such as virtual machines, to other servers in the cluster.​*
 
-3. Select **yes** to pause the server and initiate the drain process. The server status shows as **In maintenance, Draining**, and roles such as Hyper-V and VMs will immediately begin live migrating to other server(s) in the cluster. This can take a few minutes. No roles can be added to the server until it's resumed. When the draining process is finished, the server status shows as **In maintenance, Drain completed**. The operating system performs an automatic safety check to ensure it's safe to proceed. If there are unhealthy volumes, it stops and alerts you that it's not safe to proceed.
+3. Select **yes** to pause the server and initiate the drain process. The server status shows as **In maintenance, Draining**, and roles such as Hyper-V and VMs immediately begin live migrating to other server(s) in the cluster. This can take a few minutes. No roles can be added to the server until it resumes. When the draining process is finished, the server status shows as **In maintenance, Drain completed**. The operating system performs an automatic safety check to ensure it's safe to proceed. If there are unhealthy volumes, it stops and alerts you that it's not safe to proceed.
 
 ### Shut down the server
 
@@ -168,7 +168,7 @@ Resume-ClusterNode –Failback Immediate
 
 To do this in Failover Cluster Manager, go to **Nodes**, right-click the node, and then select **Resume** > **Fail Roles Back**.
 
-Once the server has resumed, it shows as **Up** in PowerShell and Failover Cluster Manager.
+Once the server resumes, it shows as **Up** in PowerShell and Failover Cluster Manager.
 
 ### Wait for storage to resync
 

azure-local/manage/maintain-servers.md

@@ -3,7 +3,7 @@ title: Set up a cluster witness
 description: Learn how to set up a cluster witness 
 author: alkohli 
 ms.topic: how-to
-ms.date: 04/29/2025
+ms.date: 05/06/2025
 ms.author: alkohli 
 ms.reviewer: stevenek 
 ---
@@ -24,7 +24,7 @@ To learn more about cluster witnesses and quorum, see [Understanding cluster and
 
 ## Before you begin
 
-Before you can create a cloud witness, you must have an Azure account and subscription, and register your Azure Stack HCI cluster with Azure. See the following articles for more information:
+Before you can create a cloud witness, you must have an Azure account and subscription, and register your Azure Stack HCI cluster with Azure. For more information, see the following articles:
 
 - Make sure that port 443 is open in your firewalls and that `*.core.windows.net` is included in any firewall allowlists you're using between the cluster and Azure Storage. For details, see [Required firewall URLs](../concepts/firewall-requirements.md).
 - If your network uses a proxy server for internet access, you must [configure proxy settings for Azure Stack HCI](./configure-proxy-settings.md).