Merge pull request #8858 from AmandaAZ/Branch-CI5668

AB#5668, AB#5252: Private PR for PR#1840

Author: Simonx Xu

support/azure/azure-kubernetes/error-codes/vmextensionerror-oraspullnetworktimeout.md

@@ -0,0 +1,52 @@
+---
+title: OrasPullNetworkTimeoutVMExtensionError When Creating AKS Clusters
+description: Learn how to troubleshoot the OrasPullNetworkTimeoutVMExtensionError error (211) when you try to create and deploy an Azure Kubernetes Service (AKS) cluster.
+ms.date: 05/09/2025
+ms.reviewer: xinhl, v-weizhu
+ms.service: azure-kubernetes-service
+#Customer intent: As an Azure Kubernetes user, I want to troubleshoot the OrasPullNetworkTimeoutVMExtensionError error code (OrasPullNetworkTimeoutVMExtensionError (211)) so that I can successfully create and deploy an Azure Kubernetes Service (AKS) cluster.
+ms.custom: sap:Create, Upgrade, Scale and Delete operations (cluster or nodepool)
+---
+# OrasPullNetworkTimeoutVMExtensionError error code (211) when deploying an AKS cluster
+
+This article discusses how to identify and resolve the `OrasPullNetworkTimeoutVMExtensionError` error (error code 211) that occurs when you try to create and deploy a Microsoft Azure Kubernetes Service (AKS) cluster.
+
+## Symptoms
+
+When you try to create an AKS cluster with the outbound type `none` or `block`, you receive the following error message:
+
+> VMExtensionProvisioningError: VM has reported a failure when processing extension 'vmssCSE'.
+>
+> Error message: "Enable failed: failed to execute command: command terminated with exit status=211
+>
+> Bootstrap Container Registry is not reachable. Please check the network configuration and try again.
+
+## Cause
+
+For [network isolated cluster](/azure/aks/concepts-network-isolated), egress traffic is limited. The feature introduces private Azure Container Registry (ACR) cache that acts as a proxy to download necessary binaries or images from Microsoft Artifact Registry (MAR) for AKS bootstrap. VM instances connect to the private ACR via a private link. Incorrect configuration of the private link causes VM bootstrap Custom Script Extension (CSE) to fail.
+
+## Solution
+
+To resolve this issue, follow these steps:
+
+1. Retrieve the ACR resource ID that AKS uses as the bootstrap ACR by running the following command:
+
+    ```console
+    az aks show -g ${RESOURCE_GROUP} -n ${CLUSTER_NAME} --query 'bootstrapProfile.containerRegistryResourceId'
+    ```
+
+2. Verify the ACR cache rule. It should include `aks-managed-rule` with source repo `mcr.microsoft.com/*` and target repo `aks-managed-reposity/*`. Ensure no other cache rule exists with source or target repo as `*`, which override `aks-managed-rule`.
+
+3. Review the [container registry private link](/azure/container-registry/container-registry-private-link) to ensure that the connection configuration is correct, including the private Domain Name System (DNS) zone and private link.
+
+4. Access any failed VM instance using Secure Shell (SSH) and run curl on the ACR host. If successful, reconcile the cluster. If it still fails, return to step 3.
+
+## References
+
+- [General troubleshooting of AKS cluster creation issues](../create-upgrade-delete/troubleshoot-aks-cluster-creation-issues.md)
+
+- [Network isolated Azure Kubernetes Service (AKS) clusters](/azure/aks/concepts-network-isolated)
+
+- [Container registry private link](/azure/container-registry/container-registry-private-link)
+
+[!INCLUDE [Azure Help Support](../../../includes/azure-help-support.md)]

support/azure/azure-kubernetes/error-codes/vmextensionerror-oraspullunauthorized.md

@@ -0,0 +1,65 @@
+---
+title: OrasPullUnauthorizedVMExtensionError When Creating AKS Clusters
+description: Learn how to troubleshoot the OrasPullUnauthorizedVMExtensionError error (212) when you try to create and deploy an Azure Kubernetes Service (AKS) cluster.
+ms.date: 05/09/2025
+ms.reviewer: xinhl, v-weizhu
+ms.service: azure-kubernetes-service
+#Customer intent: As an Azure Kubernetes user, I want to troubleshoot the OrasPullUnauthorizedVMExtensionError error code (OrasPullUnauthorizedVMExtensionError (212)) so that I can successfully create and deploy an Azure Kubernetes Service (AKS) cluster.
+ms.custom: sap:Create, Upgrade, Scale and Delete operations (cluster or nodepool)
+---
+# OrasPullUnauthorizedVMExtensionError error code (212) when deploying an AKS cluster
+
+This article discusses how to identify and resolve the `OrasPullUnauthorizedVMExtensionError` error (error code 212) that occurs when you try to create and deploy a Microsoft Azure Kubernetes Service (AKS) cluster.
+
+## Symptoms
+
+When you try to create an AKS cluster with the outbound type `none` or `block`, you receive the following error message:
+
+> VMExtensionProvisioningError: VM has reported a failure when processing extension 'vmssCSE'.
+>
+> Error message: "Enable failed: failed to execute command: command terminated with exit status=212
+>
+> Bootstrap Container Registry authorization failed. Please ensure kubelet identity has pull access to the registry.
+
+## Cause
+
+For [network isolated cluster](/azure/aks/concepts-network-isolated), egress traffic is limited. The feature introduces private Azure Container Registry (ACR) cache that acts as a proxy to download necessary binary or images from Microsoft Artifact Registry (MAR) for AKS bootstrap. It's suggested to disable anonymous access to the ACR. The AKS node uses the kubelet identity to access the ACR. If the `acrpull` permission isn't set correctly or the kubelet identity isn't bound to the VM instance, an unauthorized error occurs.
+
+## Solution
+
+To resolve this issue, follow these steps:
+
+1. Access the VM instance using Secure Shell (SSH) to get the log file`/var/log/azure/cluster-provision.log`. Review the log to determine if the issue is related to a 401 error, Azure Instance Metadata Service (IMDS) connection time-out, or an identity not found with HTTP code 400.
+
+2. Retrieve the ACR resource ID that AKS uses as the bootstrap ACR by running the following command:
+
+    ```console
+    export REGISTRY_ID=$(az aks show -g ${RESOURCE_GROUP} -n ${CLUSTER_NAME} --query 'bootstrapProfile.containerRegistryId' -o tsv)
+    ```
+
+3. If the issue is related to a 401 error, check if the kubelet identity has the `acrpull` permission to the ACR by running the following command:
+
+    ```console
+    export KUBELET_IDENTITY_PRINCIPAL_ID=$(az aks show -g ${RESOURCE_GROUP} -n ${CLUSTER_NAME} --query 'identityProfile.kubeletidentity.clientId' -o tsv)
+    ```
+
+    If not, run the following command:
+
+    ```console
+    az role assignment create --role AcrPull --scope ${REGISTRY_ID} --assignee-object-id ${KUBELET_IDENTITY_PRINCIPAL_ID} --assignee-principal-type ServicePrincipal
+    ```
+
+4. If the log error indicates that the identity isn't found, manually bind the kubelet identity to the Virtual Machine Scale Set (VMSS) for a quick fix.
+
+5. If the issue is related to IMDS connection time-out, submit a support ticket.
+6. Reconcile the cluster if the preceding operations are completed.
+
+## References
+
+- [General troubleshooting of AKS cluster creation issues](../create-upgrade-delete/troubleshoot-aks-cluster-creation-issues.md)
+
+- [Network isolated Azure Kubernetes Service (AKS) clusters](/azure/aks/concepts-network-isolated)
+
+- [container registry authentication managed identity](/azure/container-registry/container-registry-authentication-managed-identity)
+
+[!INCLUDE [Azure Help Support](../../../includes/azure-help-support.md)]

support/azure/azure-kubernetes/toc.yml

@@ -358,6 +358,10 @@
   items:
   - name: VMExtension error codes
     items:
+    - name: OrasPullNetworkTimeoutVMExtensionError
+      href: error-codes/vmextensionerror-oraspullnetworktimeout.md
+    - name: OrasPullUnauthorizedVMExtensionError
+      href: error-codes/vmextensionerror-oraspullunauthorized.md
     - name: VMExtensionError_CniDownloadTimeout error
       href: error-codes/vmextensionerror-cnidownloadtimeout.md
     - name: VMExtensionError_OutboundConnFail error