You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: AKS-Arc/tsg-upgrade-issues.md
+16-15Lines changed: 16 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,23 +4,25 @@ description: Learn how to troubleshoot and mitigate the issue when an AKS enable
4
4
ms.topic: troubleshooting
5
5
author: rcheeran
6
6
ms.author: rcheeran
7
-
ms.date: 06/25/2025
7
+
ms.date: 06/26/2025
8
8
ms.reviewer: abha
9
9
10
10
---
11
11
12
-
# Troubleshoot the issue when the AKS Arc cluster is stuck in 'Upgrading' state
12
+
# Troubleshoot AKS Arc cluster stuck in "Upgrading" state
13
13
14
-
This article describes how to fix the issue when your Azure Kubernetes Service enabled by Arc (AKS Arc) cluster is stuck in 'Upgrading' state. This issue typically occurs after updating Azure Local to version 2503 or 2504 and when you try to upgrade the Kubernetes version on your cluster.
14
+
This article describes how to fix an issue in which your Azure Kubernetes Service enabled by Arc (AKS Arc) cluster is stuck in the **Upgrading** state. This issue typically occurs after you update Azure Local to version 2503 or 2504, and you then try to upgrade the Kubernetes version on your cluster.
15
15
16
16
## Symptoms
17
17
18
-
When you try to upgrade an AKS Arc cluster, you notice that the **Current state** property of the cluster remains in the 'Upgrading' state.
18
+
When you try to upgrade an AKS Arc cluster, you notice that the **Current state** property of the cluster remains in the **Upgrading** state.
19
19
20
-
```output
20
+
```azurecli
21
21
az aksarc upgrade --name "cluster-name" --resource-group "rg-name"
22
+
```
22
23
23
-
===> Kubernetes may be unavailable during cluster upgrades.
24
+
```output
25
+
===> Kubernetes might be unavailable during cluster upgrades.
24
26
Are you sure you want to perform this operation? (y/N): y
25
27
The cluster is on version 1.28.9 and is not in a failed state.
26
28
@@ -44,10 +46,10 @@ Upgrading the AKSArc cluster. This operation might take a while...
44
46
...
45
47
```
46
48
47
-
## Possible causes and follow-ups
49
+
## Cause
48
50
49
-
- The root cause is a recent change introduced in Azure Local version 2503. Under certain conditions, if there are transient or intermittent failures during the Kubernetes upgrade process, they're not correctly detected or recovered from. This can cause the cluster state to stay stuck in the 'Upgrading' state.
50
-
- You hit this issue if the AKS Arc extension on your custom location - the `hybridaksextension` extension's version is 2.1.211 or 2.1.223. You can run the following command to check the extension version on your cluster:
51
+
- The root cause is a recent change introduced in Azure Local version 2503. Under certain conditions, if there are transient or intermittent failures during the Kubernetes upgrade process, they're not correctly detected or recovered from. This can cause the cluster state to remain in the **Upgrading** state.
52
+
- You see this issue if the AKS Arc extension on your custom location - the `hybridaksextension` extension version is 2.1.211 or 2.1.223. You can run the following command to check the extension version on your cluster:
51
53
52
54
```azurecli
53
55
az login --use-device-code --tenant <Azure tenant ID>
@@ -58,7 +60,7 @@ az k8s-extension show -g $res.HybridaksExtension.resourceGroup -c $res.ResourceB
58
60
59
61
## Mitigation
60
62
61
-
This issue can be resolved by invoking the AKS Arc update command. The `update` command retriggers the upgrade flow. You can invoke the `aksarc update` command with placeholder parameters, which do not impact the state of the cluster. So in this case, you could invoke the update call to enable NFS or SMB drivers if those features aren't already enabled. First, check if any of the storage drivers are already enabled:
63
+
You can resolve this issue by running the AKS Arc `update` command. The `update` command restarts the upgrade flow. You can run the `aksarc update` command with placeholder parameters, which do not impact the state of the cluster. So in this case, you can run the `update` command to enable NFS or SMB drivers if those features aren't already enabled. First, check if any of the storage drivers are already enabled:
62
64
63
65
```azurecli
64
66
az login --use-device-code --tenant <Azure tenant ID>
@@ -80,14 +82,14 @@ Check the storage profile section:
80
82
}
81
83
```
82
84
83
-
If one of the drivers is disabled, you can enable it using the following command:
85
+
If one of the drivers is disabled, you can enable it using one of the following commands:
84
86
85
87
```azurecli
86
88
az aksarc update --enable-smb-driver -g <resource_group_name> -n <cluster_name>
87
89
az aksarc update --enable-nfs-driver -g <resource_group_name> -n <cluster_name>
88
90
```
89
91
90
-
Running the `aksarc update` command should resolve the issue and the `Current state` parameter of the cluster should now show as 'Succeeded'. Once the status is updated, if you don't want to retain the drivers as enabled, you can revert this action by running the following command
92
+
Running the `aksarc update` command should resolve the issue and the `Current state` parameter of the cluster should now show as **Succeeded**. Once the status is updated, if you don't want to retain the drivers as enabled, you can revert this action by running the following commands:
91
93
92
94
```azurecli
93
95
az aksarc update --disable-smb-driver -g <resource_group_name> -n <cluster_name>
@@ -98,16 +100,15 @@ If both drivers are already enabled on your cluster, you can disable the one tha
98
100
99
101
## Verification
100
102
101
-
Run the following command and check that the **Current State** parameter in the JSON output is set to 'Succeeded' to confirm the K8s version upgrade is complete.
103
+
To confirm the K8s version upgrade is complete, run the following command and check that the `Current State` parameter in the JSON output is set to `Succeeded`.
102
104
103
105
```azurecli
104
106
az aksarc show -g <resource_group> -n <cluster_name>
105
-
106
107
```
107
108
108
109
## Contact Microsoft Support
109
110
110
-
If the problem persists, collect the following information before [creating a support request](aks-troubleshoot.md#open-a-support-request). Collect [AKS cluster logs](get-on-demand-logs.md) before creating the support request.
111
+
If the problem persists, collect the [AKS cluster logs](get-on-demand-logs.md)before you [create a support request](aks-troubleshoot.md#open-a-support-request).
0 commit comments