You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: support/azure/azure-kubernetes/connectivity/cannot-access-cluster-api-server-using-authorized-ip-ranges.md
+5-3Lines changed: 5 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,8 @@
1
1
---
2
2
title: Can't access the cluster API server using authorized IP ranges
3
3
description: Troubleshoot problems accessing the cluster API server when you use authorized IP address ranges in Azure Kubernetes Service (AKS).
#Customer intent: As an Azure Kubernetes user, I want to troubleshoot access issues to the cluster API server when I use authorized IP address ranges so that I can work with my Azure Kubernetes Service (AKS) cluster successfully.
@@ -14,7 +14,9 @@ This article discusses how to resolve a scenario in which you can't use authoriz
14
14
15
15
## Symptoms
16
16
17
-
If you try to create or manage an AKS cluster, you can't access the cluster API server.
17
+
If you try to create or manage resources in an AKS cluster, you can't access the cluster API server. When you run `kubectl`, you receive the following error message:
18
+
19
+
> Unable to connect to the server: dial tcp x.x.x.x:443: i/o timeout
Copy file name to clipboardExpand all lines: support/azure/azure-kubernetes/create-upgrade-delete/error-code-subnetisfull.md
+13-2Lines changed: 13 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,11 +1,11 @@
1
1
---
2
2
title: Troubleshoot the SubnetIsFull error code
3
3
description: Learn how to troubleshoot the SubnetIsFull error when you try to scale an Azure Kubernetes Service (AKS) cluster.
4
-
ms.date: 11/20/2023
4
+
ms.date: 02/28/2025
5
5
author: jotavar
6
6
ms.author: jotavar
7
7
editor: v-jsitser
8
-
ms.reviewer: rissing, chiragpa, v-leedennis
8
+
ms.reviewer: rissing, chiragpa, addobres
9
9
ms.service: azure-kubernetes-service
10
10
#Customer intent: As an Azure Kubernetes user, I want to troubleshoot the SubnetIsFull error code so that I can successfully scale an Azure Kubernetes Service (AKS) cluster.
11
11
ms.custom: sap:Create, Upgrade, Scale and Delete operations (cluster or nodepool)
@@ -45,6 +45,17 @@ Trying to update a subnet's Classless Inter-Domain Routing (CIDR) address space
45
45
46
46
4. Delete the original node pool by running the [az aks nodepool delete](/cli/azure/aks/nodepool#az-aks-nodepool-delete) command.
47
47
48
+
49
+
## Best practices
50
+
51
+
To avoid `SubnetIsFull` issues in Azure Kubernetes Service (AKS), follow best practices that are related to subnet sizing, IP address management, and node pool strategies. Here are some key recommendations:
52
+
53
+
- Plan for Future Growth: When you create subnets, make sure that they're large enough to accommodate future growth. We recommend that you reserve more IP addresses than you currently need to avoid running out of space as the cluster scales.
54
+
- Use Larger Subnet CIDR: If possible, use a larger subnet CIDR to provide more IP addresses. This strategy helps to accommodate more nodes and pods without running into IP exhaustion issues.
55
+
- Monitor IP Usage: To identify potential issues before they become critical, regularly monitor the IP address usage within your subnets. Tools such as Azure Monitor can help track IP address consumption.
56
+
- Optimize IP Allocation: Make sure that IP addresses are allocated efficiently. Avoid reserving IP addresses unnecessarily. To free up space, release any unused IP addresses.
57
+
- Use multiple node pools: Consider using node pools that have different subnets to distribute the IP address load. This strategy can help mitigate the risk of running out of IP addresses in a single subnet.
58
+
48
59
## More information
49
60
50
61
-[General troubleshooting of AKS cluster creation issues](troubleshoot-aks-cluster-creation-issues.md)
Copy file name to clipboardExpand all lines: support/azure/azure-kubernetes/error-codes/unsatisfiablepdb-error.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,8 @@
1
1
---
2
2
title: AKS cluster upgrade fails with UnsatisfiablePDB error
3
3
description: Provides solutions to the UnsatisfiablePDB error when you try to upgrade an Azure Kubernetes Service (AKS) cluster.
4
-
ms.date: 10/27/2023
5
-
ms.reviewer: chiragpa, v-weizhu
4
+
ms.date: 03/05/2025
5
+
ms.reviewer: chiragpa, v-weizhu, addobres
6
6
ms.service: azure-kubernetes-service
7
7
ms.custom: sap:Create, Upgrade, Scale and Delete operations (cluster or nodepool)
8
8
#Customer intent: As an Azure Kubernetes Services (AKS) user, I want to troubleshoot an Azure Kubernetes Service cluster upgrade that failed because of a UnsatisfiablePDB error so that I can upgrade the cluster successfully.
Copy file name to clipboardExpand all lines: support/azure/azure-kubernetes/extensions/cannot-pull-image-from-acr-to-aks-cluster.md
+32-7Lines changed: 32 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
---
2
2
title: Can't pull images from Azure Container Registry to Kubernetes
3
3
description: This article helps you troubleshoot the most common errors that you may encounter when pulling images from a container registry to an AKS cluster.
@@ -58,6 +58,8 @@ The following sections help you troubleshoot the most common errors that are dis
58
58
59
59
## Cause 1: 401 Unauthorized error
60
60
61
+
### <aid="cause1a"></a>Cause 1a: 401 Unauthorized error due to incorrect authorization
62
+
61
63
An AKS cluster requires an identity. This identity can be either a managed identity or a service principal. If the AKS cluster uses a managed identity, the kubelet identity is used for authenticating with ACR. If the AKS cluster is using as an identity a service principal, the service principal itself is used for authenticating with ACR. No matter what the identity is, the proper authorization that's used to pull an image from a container registry is necessary. Otherwise, you may get the following "401 Unauthorized" error:
62
64
63
65
> Failed to pull image "\<acrname>.azurecr.io/\<repository\:tag>": [rpc error: code = Unknown desc = failed to pull and unpack image "\<acrname>.azurecr.io/\<repository\:tag>": failed to resolve reference "\<acrname>.azurecr.io/\<repository\:tag>": failed to authorize: failed to fetch oauth token: **unexpected status: 401 Unauthorized**
@@ -70,7 +72,7 @@ Several solutions can help you resolve this error, subject to the following cons
70
72
71
73
- Solutions [5][cause1-solution5] and [6][cause1-solution6] are applicable for the Kubernetes method of [pulling a Kubernetes secret](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/).
72
74
73
-
### Solution 1: Make sure AcrPull role assignment is created for identity
75
+
####Solution 1: Make sure AcrPull role assignment is created for identity
74
76
75
77
The integration between AKS and Container Registry creates an AcrPull role assignment at container registry level for the AKS cluster's kubelet identity. Make sure that the role assignment is created.
76
78
@@ -92,7 +94,7 @@ If the AcrPull role assignment isn't created, create it by [configuring Containe
92
94
az aks update -n <myAKSCluster> -g <myResourceGroup> --attach-acr <acr-resource-id>
93
95
```
94
96
95
-
### Solution 2: Make sure service principal isn't expired
97
+
####Solution 2: Make sure service principal isn't expired
96
98
97
99
Make sure that the secret of the service principal that's associated with the AKS cluster isn't expired. To check the expiration date of your service principal, run the following commands:
98
100
@@ -107,7 +109,7 @@ For more information, see [Check the expiration date of your service principal](
107
109
108
110
If the secret is expired, [update the credentials for the AKS cluster](/azure/aks/update-credentials).
109
111
110
-
### Solution 3: Make sure AcrPull role is assigned to correct service principal
112
+
####Solution 3: Make sure AcrPull role is assigned to correct service principal
111
113
112
114
In some cases, the container registry role assignment still refers to the old service principal. For example, when the service principal of the AKS cluster is replaced with a new one. To make sure that the container registry role assignment refers to the correct service principal, follow these steps:
113
115
@@ -128,7 +130,7 @@ In some cases, the container registry role assignment still refers to the old se
128
130
129
131
1. Compare the two service principals. If they don't match, integrate the AKS cluster with the container registry again.
130
132
131
-
### Solution 4: Make sure the kubelet identity is referenced in the AKS VMSS
133
+
#### Solution 4: Make sure the kubelet identity is referenced in the AKS VMSS
132
134
133
135
When a managed identity is used for authentication with the ACR, the managed identity is known as the kubelet identity. By default, the kubelet identity is assigned at the AKS VMSS level. If the kubelet identity is removed from the AKS VMSS, the AKS nodes can't pull images from the ACR.
134
136
@@ -155,7 +157,7 @@ Because modifications to the AKS VMSS aren't supported, they don't propagate at
155
157
az aks update --resource-group <MyResourceGroup> --name <MyManagedCluster>
156
158
```
157
159
158
-
### Solution 5: Make sure the service principal is correct and the secret is valid
160
+
####Solution 5: Make sure the service principal is correct and the secret is valid
159
161
160
162
If you pull an image by using an [image pull secret](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/), and that Kubernetes secret was created by using the values of a service principal, make sure that the associated service principal is correct and the secret is still valid. Follow these steps:
161
163
@@ -179,7 +181,7 @@ If you pull an image by using an [image pull secret](https://kubernetes.io/docs/
179
181
180
182
1. Update or re-create the Kubernetes secret accordingly.
181
183
182
-
### Solution 6: Make sure the Kubernetes secret has the correct values of the container registry admin account
184
+
####Solution 6: Make sure the Kubernetes secret has the correct values of the container registry admin account
183
185
184
186
If you pull an image by using an [image pull secret](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/), and that Kubernetes secret was created by using values of [container registry admin account](/azure/container-registry/container-registry-authentication#admin-account), make sure that the values in the Kubernetes secret are the same as the values of the container registry admin account. Follow these steps:
185
187
@@ -202,6 +204,29 @@ If you pull an image by using an [image pull secret](https://kubernetes.io/docs/
202
204
> [!NOTE]
203
205
> If a **Regenerate** password operation occurred, an operation that's named "Regenerate Container Registry Login Credentials" will be displayed in the **Activity log** page of the container registry. The **Activity log** has a [90-day retention period](/azure/azure-monitor/essentials/activity-log#retention-period).
204
206
207
+
### Cause 1b: 401 Unauthorized error due to incompatible architecture
208
+
209
+
You might encounter a "401 Unauthorized" error even when the AKS cluster identity is authorized (as described in the [Cause 1a: 401 Unauthorized error due to incorrect authorization](#cause1a) section). This issue can happen if the container image in the ACR doesn't match the architecture (such as arm64 versus amd64) of the node running the container. For example, deploying an arm64 image on an amd64 node or vice versa can result in this error.
210
+
211
+
The error message will appear as follows:
212
+
213
+
> Failed to pull image "\<acrname>.azurecr.io/\<repository:\tag>": [rpc error: code = NotFound desc = failed to pull and unpack image "\<acrname>.azurecr.io/\<repository:\tag>": no match for platform in manifest: not found, failed to pull and unpack image "\<acrname>.azurecr.io/\<repository\:tag>": failed to resolve reference "\<acrname>.azurecr.io/\<repository\:tag>": failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://\<acrname>.azurecr.io/oauth2/token?scope=repository%3A\<repository>%3Apull&service=\<acrname>.azurecr.io: 401 Unauthorized]
214
+
215
+
When diagnosing this issue using the Azure CLI, you might see an unexpected "exec format error" if your system node pool runs a different architecture than the image in the ACR:
216
+
217
+
```azurecli
218
+
az aks check-acr --resource-group <MyResourceGroup> --name <MyManagedCluster> --acr <myacr>.azurecr.io
219
+
220
+
exec /canipull: exec format error
221
+
```
222
+
223
+
#### Solution: Push images with the correct architecture or push multi-architecture images
224
+
225
+
To resolve this issue, use one of the following methods:
226
+
227
+
- Ensure the container images pushed to ACR match the architecture of your AKS nodes (for example, arm64 or amd64).
228
+
- Create and push multi-architecture images that support both arm64 and amd64 architectures.
229
+
205
230
## Cause 2: Image not found error
206
231
207
232
> Failed to pull image "\<acrname>.azurecr.io/\<repository\:tag>": [rpc error: code = NotFound desc = failed to pull and unpack image "\<acrname>.azurecr.io/\<repository\:tag>": failed to resolve reference "\<acrname>.azurecr.io/\<repository\:tag>": **\<acrname>.azurecr.io/\<repository\:tag>: not found**
0 commit comments