Merge pull request #267535 from AbhishekMallick-MS/main

AKS backup - monitor warning and troubleshoot

Author: v-dirichards

articles/backup/azure-kubernetes-service-backup-troubleshoot.md

@@ -2,7 +2,7 @@
 title: Troubleshoot Azure Kubernetes Service backup
 description: Symptoms, causes, and resolutions of the Azure Kubernetes Service backup and restore operations.
 ms.topic: troubleshooting
-ms.date: 12/28/2023
+ms.date: 02/28/2024
 ms.service: backup
 ms.custom:
   - ignite-2023
@@ -222,6 +222,88 @@ This error code can appear while you enable AKS backup to store backups in a vau
 
 3. Create a backup policy for operational tier backup (only snapshots for the AKS cluster).
 
+## AKS backup and restore jobs completed with warnings
+
+### UserErrorPVSnapshotDisallowedByPolicy
+
+**Error code**: UserErrorPVSnapshotDisallowedByPolicy
+
+**Cause**: An Azure policy is assigned over subscription that ceases the CSI driver to take the volume snapshot.
+
+**Recommended action**: Remove the Azure Policy ceasing the disk snapshot operation, and then perform an on-demand backup.
+
+### UserErrorPVSnapshotLimitReached
+
+**Error code**: UserErrorPVSnapshotLimitReached
+
+**Cause**: There is a limited number of snapshots for a Persistent Volume that can exist at a point-in-time. For Azure Disk-based Persistent Volumes, the limit is *500 snapshots*. This error appears when snapshots for specific Persistent Volumes aren't taken due to existence of snapshots higher than the supported limits.
+
+**Recommended action**: Update the Backup Policy to reduce the retention duration and wait for older recovery points to be deleted by the Backup vault.
+
+### CSISnapshottingTimedOut
+
+**Error code**: CSISnapshottingTimedOut
+
+**Cause**: Snapshot has failed because CSI Driver is getting timed out to fetch the snapshot handle.  
+
+**Recommended action**: Review the logs and retry the operation to get successful snapshots by running an on-demand backup, or wait for next scheduled backup.
+
+### UserErrorHookExecutionFailed
+
+**Error code**: UserErrorHookExecutionFailed
+
+**Cause**: When hooks applied to run along with backups and restores have encountered an error, and aren't successfully applied.
+
+**Recommended action**: Review the logs, update the hooks, and then retry backup/restore operation.
+
+### UserErrorNamespaceNotFound
+
+**Error code**: UserErrorNamespaceNotFound
+
+**Cause**: Namespaces provided in Backup Configuration is missing while performing backups. Either the namespace was wrongly provided or has been deleted.
+
+**Recommended action**: Check if the Namespaces to be backed up are correctly provided.
+
+### UserErrorPVCHasNoVolume
+
+**Error code**: UserErrorPVCHasNoVolume
+
+**Cause**: The Persistent Volume Claim (PVC) in context does not have a Persistent Volume attached to it. So, the PVC will not be backed up.
+
+**Recommended action**: Attach a volume to the PVC, if it needs to be backed up.
+
+### UserErrorPVCNotBoundToVolume
+
+**Error code**: UserErrorPVCNotBoundToVolume
+
+**Cause**: The PVC in context is in *Pending* state and doesn't have a Persistent Volume attached to it. So, the PVC will not be backed up. 
+
+**Recommended action**: Attach a volume to the PVC, if it needs to be backed up.
+
+### UserErrorPVNotFound
+
+**Error code**: UserErrorPVNotFound
+
+**Cause**: The underlying storage medium for the Persistent Volume is missing. 
+
+**Recommended action**: Check and attached a new Persistent Volume with actual storage medium attached.
+
+### UserErrorStorageClassMissingForPVC
+
+**Error code**: UserErrorStorageClassMissingForPVC
+
+**Cause**: AKS backup checks for the storage class being used and skips the Persistent Volume from taking snapshots due to unavailability of the class. 
+
+**Recommended action**: Update the PVC specifications with the storage class used.
+
+### UserErrorSourceandTargetClusterCRDVersionMismatch
+
+**Error code**: UserErrorSourceandTargetClusterCRDVersionMismatch
+
+**Cause**: The source AKS cluster and Target AKS cluster during restore have different versions of *FlowSchema* and *PriorityLevelConfigurations CRs*. Some Kubernetes resources aren't restored due to the mismatch in cluster versions.
+
+**Recommended action**: Use same cluster version for Target cluster as Source cluster or manually apply the CRs.
+
 ## Next steps
 
 - [About Azure Kubernetes Service (AKS) backup](azure-kubernetes-service-backup-overview.md)

articles/backup/azure-kubernetes-service-cluster-manage-backups.md

@@ -6,7 +6,7 @@ ms.service: backup
 ms.custom:
   - devx-track-azurecli
   - ignite-2023
-ms.date: 02/27/2024
+ms.date: 02/28/2024
 author: AbhishekMallick-MS
 ms.author: v-abhmallick
 ---
@@ -139,6 +139,22 @@ To enable Trusted Access between Backup vault and AKS cluster, use the following
 
 Learn more about [other commands related to Trusted Access](../aks/trusted-access-feature.md#trusted-access-feature-overview).
 
+## Monitor AKS backup jobs completed with warnings
+
+When a scheduled or an on-demand backup or restore operation is performed, a job is created corresponding to the operation to track its progress. In case of a failure, these jobs allow you to identify error codes and fix issues to run a successful job later. 
+
+For AKS backup, backup and restore jobs can show the status **Completed with Warnings**. This status appears when the backup and restore operation isn't fully successful due to issues in user-defined configurations or internal state of the workload.
+
+:::image type="content" source="./media/azure-kubernetes-service-cluster-manage-backups/backup-restore-jobs-completed-with-warnings.png" alt-text="Screenshot shows the backup and restore jobs completed with warnings." lightbox="./media/azure-kubernetes-service-cluster-manage-backups/backup-restore-jobs-completed-with-warnings.png":::
+
+For example, if a backup job for an AKS cluster completes with the status **Completed with Warnings**, a restore point will be created, but it might not have been able to back up all the resources in the cluster as per the backup configuration. The job will show warning details, providing the *issues* and *resources* that were impacted during the operation. 
+
+To view these warnings, select **View Details** next to **Warning Details**.
+
+:::image type="content" source="./media/azure-kubernetes-service-cluster-manage-backups/example-backup-job-with-warning-details.png" alt-text="Screenshot shows the job warming details." lightbox="./media/azure-kubernetes-service-cluster-manage-backups/example-backup-job-with-warning-details.png"::: 
+
+Learn [how to identify and resolve the error](azure-kubernetes-service-backup-troubleshoot.md#aks-backup-extension-installation-error-resolutions). 
+
 ## Next steps
 
 - [Back up Azure Kubernetes Service cluster](azure-kubernetes-service-cluster-backup.md)