Skip to content

Commit e7982bf

Browse files
committed
Add TSG for cluster upgrade
1 parent 5d95c4d commit e7982bf

File tree

1 file changed

+11
-9
lines changed

1 file changed

+11
-9
lines changed

AKS-Arc/tsg-aksarc-upgrade-issues.md

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,11 @@ ms.reviewer: abha
1111

1212
# Troubleshoot the issue when the AKS Arc cluster is stuck in 'Upgrading' state
1313

14-
This article describes how to fix the issue when your Azure Kubernetes Service enablied by Arc (AKS Arc) cluster is stuck in 'Upgrading' state. This issue typically occurs after updating Azure Local to version 2503 or 2504 and when you try to upgrade the Kubernetes version on your cluster.
14+
This article describes how to fix the issue when your Azure Kubernetes Service enabled by Arc (AKS Arc) cluster is stuck in 'Upgrading' state. This issue typically occurs after updating Azure Local to version 2503 or 2504 and when you try to upgrade the Kubernetes version on your cluster.
1515

1616
## Symptoms
1717

18-
When you try to upgrade an AKS Arc cluster, you notice that the **Current state** property of the cluster remains in the 'Upgrading' state.
18+
When you try to upgrade an AKS Arc cluster, you notice that the **Current state** property of the cluster remains in the 'Upgrading' state.
1919

2020
```output
2121
az aksarc upgrade --name "cluster-name" --resource-group "rg-name"
@@ -34,17 +34,16 @@ Upgrading the AKSArc cluster. This operation might take a while...
3434
"id": "/subscriptions/fbaf508b-cb61-4383-9cda-a42bfa0c7bc9/resourceGroups/Bellevue/providers/Microsoft.Kubernetes/ConnectedClusters/Bel-cluster/providers/Microsoft.HybridContainerService/ProvisionedClusterInstances/default",
3535
"name": "default",
3636
"properties": {
37-
"kubernetesVersion": "1.30.4",
38-
"provisioningState": "Succeeded",
39-
"currentState": "Upgrading",
37+
"kubernetesVersion": "1.30.4",
38+
"provisioningState": "Succeeded",
39+
"currentState": "Upgrading",
4040
"errorMessage": null,
4141
"operationStatus": null
4242
"agentPoolProfiles": [
4343
{
4444
...
4545
```
4646

47-
4847
## Possible causes and follow-ups
4948

5049
- The root cause is a recent change introduced in Azure Local version 2503. Under certain conditions, if there are transient or intermittent failures during the Kubernetes upgrade process, they're not correctly detected or recovered from. This can cause the cluster state to stay stuck in the 'Upgrading' state.
@@ -57,16 +56,18 @@ $res=get-archcimgmt
5756
az k8s-extension show -g $res.HybridaksExtension.resourceGroup -c $res.ResourceBridge.name --cluster-type appliances --name hybridaksextension
5857
```
5958

60-
6159
## Mitigation
62-
This issue can be resolved by invoking the AKS Arc update command. The `update` command will retriggers the upgrade flow. You can invoke the `aksarc update` command with placeholder parameters, which dont impact the state of the cluster. So in this case, you could invoke the update call to enable NFS or SMB drivers if those features aren't already enabled. First, check if any of the storage drivers are already enabled:
60+
61+
This issue can be resolved by invoking the AKS Arc update command. The `update` command retriggers the upgrade flow. You can invoke the `aksarc update` command with placeholder parameters, which do not impact the state of the cluster. So in this case, you could invoke the update call to enable NFS or SMB drivers if those features aren't already enabled. First, check if any of the storage drivers are already enabled:
6362

6463
```azurecli
6564
az login --use-device-code --tenant <Azure tenant ID>
6665
az account set -s <subscription ID>
6766
az aksarc show -g <resource_group_name> -n <cluster_name>
6867
```
68+
6969
Check the storage profile section:
70+
7071
```json
7172
"storageProfile": {
7273
"nfsCsiDriver": {
@@ -92,17 +93,18 @@ Running the `aksarc update` command should resolve the issue and the `Current st
9293
az aksarc update --disable-smb-driver -g <resource_group_name> -n <cluster_name>
9394
az aksarc update --disable-nfs-driver -g <resource_group_name> -n <cluster_name>
9495
```
96+
9597
If both drivers are already enabled on your cluster, you can disable the one that is not in use. If you require both drivers to remain enabled, contact Microsoft Support for further assistance.
9698

9799
## Verification
100+
98101
Run the following command and check that the **Current State** parameter in the JSON output is set to 'Succeeded' to confirm the K8s version upgrade is complete.
99102

100103
```azurecli
101104
az aksarc show -g <resource_group> -n <cluster_name>
102105
103106
```
104107

105-
106108
## Contact Microsoft Support
107109

108110
If the problem persists, collect the following information before [creating a support request](aks-troubleshoot.md#open-a-support-request). Collect [AKS cluster logs](get-on-demand-logs.md) before creating the support request.

0 commit comments

Comments
 (0)