Velero backup to Azure can't sync backups back into Velero and backups are stuck in 'Finalizing'

[agelogos]

Hello,

I am trying to use Velero to store backups in Azure using the "velero-plugin-for-microsoft-azure" plugin.

I set up both Velero and the plugin using Helm managed by Argo. Below is the values file that I use:

values.txt

The logs around syncing backups and backing up are here:

logs.txt

Using 'kubectl -n velero logs deployment/velero | grep error' I get these additional logs:

error-logs.txt

The issue seems to be the plugin silently exiting because of some error, but I can't find anywhere that the configuration is incorrect.

The workloadIdentity looks like it works perfectly fine as snapshots are created in the correct place and backups are as well. However, the backups are stuck in 'Finalizing' and using 'kubectl get backups -n velero' returns nothing. The json file for one of these backups in Azure can be seen below:

backup.json

I hope you can help me, this has stumped me for a while! Please let me know if you require any additional information.

Thank you!

[agelogos]

A small update:

Manual backups seem to be working correctly, but the schedules are unable to finalize.

[agelogos]

Closing this..

It was caused by how ArgoCD and Velero interact. When using scheduled backups Argo would delete the backups instantly unless this was configured:

      useOwnerReferencesInBackup: true

However, enabling that would cause the backups to be affected whenever the schedule was updated or deleted, essentially triggering deletions of backups if you were to ever reconfigure a schedule.

The solution that ended up working was to add this to the excludedResources in the ArgoCD values.yaml:

configs:
    cm:
      resource.exclusions: |
        ### Remember to also configure the default exclusions here or you will override them.
        - apiGroups:
          - velero.io
          kinds:
          - Backup
          - Restore
          clusters:
          - "*"

This essentially allows the backup and restore custom resources to live as orphaned resources without being pruned by Argo - allowing Velero to handle their lifetimes.

[kaovilai]

fyi we already doc some of this here https://velero.io/docs/v1.15/backup-reference/#backups-ownerreference-with-schedule

[agelogos]

Thank you, I've completely missed that. I'll have a look to see if my solution aligns.