Skip to content

Commit ffe806d

Browse files
author
AbhishekMallick01
committed
more updates
1 parent 0d68cc3 commit ffe806d

File tree

1 file changed

+44
-15
lines changed

1 file changed

+44
-15
lines changed

articles/backup/azure-kubernetes-service-backup-troubleshoot.md

Lines changed: 44 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,14 @@ This article provides troubleshooting steps that help you resolve Azure Kubernet
1414

1515
## AKS Backup Extension installation error resolutions
1616

17-
### Error 1
17+
### Scenario 1
18+
19+
**Error message**:
20+
21+
```Erroe
22+
{Helm installation from path [] for release [azure-aks-backup] failed with the following error: err [release azure-aks-backup failed, and has been uninstalled due to atomic being set: failed post-install: timed out waiting for the condition]} occurred while doing the operation: {Installing the extension} on the config"`
23+
```
1824

19-
**Error message**: `{Helm installation from path [] for release [azure-aks-backup] failed with the following error: err [release azure-aks-backup failed, and has been uninstalled due to atomic being set: failed post-install: timed out waiting for the condition]} occurred while doing the operation: {Installing the extension} on the config"`
2025

2126
**Cause**: The extension has been installed successfully, but the pods aren't spawning. This happens because the required compute and memory aren't available for the pods.
2227

@@ -28,11 +33,15 @@ To scale node pool on Azure portal, follow these steps:
2833
1. Select **Scale node pool**, and then update the *minimum* and *maximum* values on the **Node count range**.
2934
1. Select **Apply**.
3035

31-
### Error 2
36+
### Scenario 2
3237

33-
Error message: `BackupStorageLocation "default" is unavailable: rpc error: code = Unknown desc = azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/e30af180-aa96-4d81-981a-b67570b0d615/resourceGroups/AzureBackupRG_westeurope_1/providers/Microsoft.Storage/storageAccounts/devhayyabackup/listKeys?%24expand=kerb&api-version=2019-06-01: StatusCode=404 -- Original Error: adal: Refresh request failed. Status Code = '404'. Response body: no azure identity found for request clientID 4e95##### REDACTED #####0777`
38+
**Error message**:
3439

35-
`Endpoint http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&client_id=4e95dcc5-a769-4745-b2d9-`
40+
```Error
41+
BackupStorageLocation "default" is unavailable: rpc error: code = Unknown desc = azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/e30af180-aa96-4d81-981a-b67570b0d615/resourceGroups/AzureBackupRG_westeurope_1/providers/Microsoft.Storage/storageAccounts/devhayyabackup/listKeys?%24expand=kerb&api-version=2019-06-01: StatusCode=404 -- Original Error: adal: Refresh request failed. Status Code = '404'. Response body: no azure identity found for request clientID 4e95##### REDACTED #####0777`
42+
43+
Endpoint http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&client_id=4e95dcc5-a769-4745-b2d9-
44+
```
3645

3746
**Cause**: When you enable pod-managed identity on your AKS cluster, an *AzurePodIdentityException* named *aks-addon-exception* is added to the *kube-system* namespace. An *AzurePodIdentityException* allows pods with certain labels to access the Azure Instance Metadata Service (IMDS) endpoint without being intercepted by the NMI server.
3847

@@ -42,20 +51,30 @@ The extension pods aren't exempt, and require the Azure Active Directory (Azure
4251

4352
1. Run the following command:
4453

45-
`az aks pod-identity exception add --resource-group shracrg --cluster-name shractestcluster --namespace dataprotection-microsoft --pod-labels app.kubernetes.io/name=dataprotection-microsoft-kubernetes`
54+
55+
```azurepowershell-interactive
56+
az aks pod-identity exception add --resource-group shracrg --cluster-name shractestcluster --namespace dataprotection-microsoft --pod-labels app.kubernetes.io/name=dataprotection-microsoft-kubernetes
57+
```
4658

4759
2. To verify *Azurepodidentityexceptions* in cluster, run the following command:
4860

49-
`kubectl get Azurepodidentityexceptions --all-namespaces`
61+
```azurepowershell-interactive
62+
kubectl get Azurepodidentityexceptions --all-namespaces
63+
```
5064

5165
3. To assign the *Storage Account Contributor* role to the extension identity, run the following command:
5266

53-
`az role assignment create --assignee-object-id $(az k8s-extension show --name azure-aks-backup --cluster-name aksclustername --resource-group aksclusterresourcegroup --cluster-type managedClusters --query aksAssignedIdentity.principalId --output tsv) --role 'Storage Account Contributor' --scope /subscriptions/subscriptionid/resourceGroups/storageaccountresourcegroup/providers/Microsoft.Storage/storageAccounts/storageaccountname`
67+
```azurepowershell-interactive
68+
az role assignment create --assignee-object-id $(az k8s-extension show --name azure-aks-backup --cluster-name aksclustername --resource-group aksclusterresourcegroup --cluster-type managedClusters --query aksAssignedIdentity.principalId --output tsv) --role 'Storage Account Contributor' --scope /subscriptions/subscriptionid/resourceGroups/storageaccountresourcegroup/providers/Microsoft.Storage/storageAccounts/storageaccountname
69+
```
5470

55-
### Error 3
71+
### Scenario 3
5672

57-
**Error message**: `{"Message":"Error in the getting the Configurations: error {Post \https://centralus.dp.kubernetesconfiguration.azure.com/subscriptions/ subscriptionid /resourceGroups/ aksclusterresourcegroup /provider/managedclusters/clusters/ aksclustername /configurations/getPendingConfigs?api-version=2021-11-01\: dial tcp: lookup centralus.dp.kubernetesconfiguration.azure.com on 10.63.136.10:53: no such host}","LogType":"ConfigAgentTrace","LogLevel":"Error","Environment":"prod","Role":"ClusterConfigAgent","Location":"centralus","ArmId":"/subscriptions/ subscriptionid /resourceGroups/ aksclusterresourcegroup /providers/Microsoft.ContainerService/managedclusters/ aksclustername ","CorrelationId":"","AgentName":"ConfigAgent","AgentVersion":"1.8.14","AgentTimestamp":"2023/01/19 20:24:16"}`
73+
**Error message**:
5874

75+
```Error
76+
{"Message":"Error in the getting the Configurations: error {Post \https://centralus.dp.kubernetesconfiguration.azure.com/subscriptions/ subscriptionid /resourceGroups/ aksclusterresourcegroup /provider/managedclusters/clusters/ aksclustername /configurations/getPendingConfigs?api-version=2021-11-01\: dial tcp: lookup centralus.dp.kubernetesconfiguration.azure.com on 10.63.136.10:53: no such host}","LogType":"ConfigAgentTrace","LogLevel":"Error","Environment":"prod","Role":"ClusterConfigAgent","Location":"centralus","ArmId":"/subscriptions/ subscriptionid /resourceGroups/ aksclusterresourcegroup /providers/Microsoft.ContainerService/managedclusters/ aksclustername ","CorrelationId":"","AgentName":"ConfigAgent","AgentVersion":"1.8.14","AgentTimestamp":"2023/01/19 20:24:16"}`
77+
```
5978
**Cause**: Specific FQDN/application rules are required to use cluster extensions in the AKS clusters. [Learn more](/azure/aks/limit-egress-traffic#cluster-extensions).
6079

6180
This error appears due to absence of these FQDN rules because of which configuration information from the Cluster Extensions service wasn't available.
@@ -64,23 +83,33 @@ This error appears due to absence of these FQDN rules because of which configura
6483

6584
1. To fetch *Existing CoreDNS-custom* YAML in your cluster (save it on your local for reference later), run the following command:
6685

67-
`kubectl get configmap coredns-custom -n kube-system -o yaml`
86+
```azurepowershell-interactive
87+
kubectl get configmap coredns-custom -n kube-system -o yaml
88+
```
6889

6990
2. To override mapping for *Central US DP* endpoint to public IP (download the YAML file attached), run the following command:
7091

71-
`kubectl apply -f corednsms.yaml`
92+
```azurepowershell-interactive
93+
kubectl apply -f corednsms.yaml
94+
```
7295

7396
3. To force reload `coredns` pods, run the following command:
7497

75-
`kubectl delete pod --namespace kube-system -l k8s-app=kube-dns`
98+
```azurepowershell-interactive
99+
kubectl delete pod --namespace kube-system -l k8s-app=kube-dns
100+
```
76101

77102
4. To perform `NSlookup` from the *ExtensionAgent* pod to check if *coreDNS-custom* is working, run the following command:
78103

79-
`kubectl exec -i -t pod/extension-agent-<pod guid that's there in your cluster> -n kube-system -- nslookup centralus.dp.kubernetesconfiguration.azure.com`
104+
```azurepowershell-interactive
105+
kubectl exec -i -t pod/extension-agent-<pod guid that's there in your cluster> -n kube-system -- nslookup centralus.dp.kubernetesconfiguration.azure.com
106+
```
80107

81108
5. To check logs of the *ExtensionAgent* pod, run the following command:
82109

83-
`kubectl logs pod/extension-agent-<pod guid that’s there in your cluster> -n kube-system --tail=200`
110+
```azurepowershell-interactive
111+
kubectl logs pod/extension-agent-<pod guid that’s there in your cluster> -n kube-system --tail=200
112+
```
84113

85114
6. Delete and reinstall Backup Extension to initiate backup.
86115

0 commit comments

Comments
 (0)