redis-ha upgrade tries to modify immutable field

[jmmclean]

Describe the bug

The change from DandyDeveloper/charts#342 creates an issue where an immutable field is attempting to be changed:

Failed to compare desired state to live state: failed to calculate diff: error calculating server side diff: serverSideDiff error: error running server side apply in dryrun mode for resource Deployment/argocd-redis-ha-haproxy: Deployment.apps "argocd-redis-ha-haproxy" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:v1.LabelSelectorRequirement(nil)}: field is immutable

Normally, I would expect argocd to rollout what it can then do a replace on the CR having this issue, but with SSA it goes into an unknown state so there is not a great way to reconcile this without doing heavy manual intervention.

Related helm chart

argo-cd

Helm chart version

9.1.0

To Reproduce

1. deploy v8.3.5 of argocd helm chart
2. deploy v 9.1.0 of argocd helm chart

Expected behavior

ArgoCD should rollout changes as expected, however it cannot because the redis-ha chart changed matchLabels selector, which is an immutable field.

Screenshots

Additional context

No response

[jmmclean]

Noted in an old version of a readme, later removed (I guess)

https://github.com/DandyDeveloper/charts/blob/a03b6a6f4d72b6606ce9a218c7d0026350b48ad0/charts/redis-ha/README.md#4341---upgrade-may-complain-about-selector-label-changes-being-immutable

[mkilchhofer]

Oh you are right, the subchart we use for redis-ha had a "breaking change" within a minor or patch version and it got picked up during the 3.2 release here:

• chore(argo-cd): Update dependency argoproj/argo-cd to v3.2.0 #3568

Could you solve it in the meantime?

I see two options:

• manually deleting the Deployment in Argo CD by using the "non-cascading" deletion method:

  

• Temporary switch back to the Redis single by setting

  redis-ha:
    enabled: false

  And then after everything is synced, switch back to true

If either of that works, we can place a quick note in the changelog section maybe you want to contribute that notice? :)

[jmmclean]

Hey @mkilchhofer I was able to get around this by doing a couple things; however it should be noted that ServerSide diffing fails because of this issue argoproj/argo-cd#22562. So I had to do the below steps:

1. Disable Server Side Diff (Re enable ClientSide Diff)
2. Do a sync after client side diff was merged(let it fail bc of redis). NOTE: I had to do step 3 for a sync to occur, otherwise ArgoCD couldnt update itself bc of server side diffing failure.
3. Delete the redis deployment by orphaning it (this meant ArgoCD did a sync bc of autosync being enabled with the new deployment
4. Manually cleanup the old ReplicaSets after the new redis deployment was rolled out

FWIW - Im not rolling this out in our production environment because of a regression here argoproj/argo-cd#25184. waiting on 3.2.1 to be cut :)

[jmmclean]

Besides the helm chart CI job failing bc it thinks a doc change requires a version bump, I made a PR for the changelog reference you requested #3582