Metal3 chart add upgrade CI coverage

[hardys]

I tried upgrading from 0.6.1 to 0.6.4 and it fails - looks like an upgrade issue related to #98 which changed the selectorLabels for each component

We need to decide if those additions are critical or can be reverted (but if we do that upgrade from 0.6.3 and 0.6.4 won't work AFAICS), and also we should consider adding CI coverage to test helm upgrade from the current to proposed release.

helm upgrade metal3 suse-edge/metal3 --version 0.6.4 --namespace metal3-system -f /home/metal3/helm-values.yaml 
Error: UPGRADE FAILED: cannot patch "baremetal-operator-controller-manager" with kind Deployment: Deployment.apps "baremetal-operator-controller-manager" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/component":"baremetal-operator", "app.kubernetes.io/instance":"metal3", "app.kubernetes.io/name":"manger", "control-plane":"controller-manager"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable && cannot patch "metal3-metal3-ironic" with kind Deployment: Deployment.apps "metal3-metal3-ironic" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/component":"ironic", "app.kubernetes.io/instance":"metal3", "app.kubernetes.io/name":"metal3-ironic"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable && cannot patch "metal3-metal3-mariadb" with kind Deployment: Deployment.apps "metal3-metal3-mariadb" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/component":"mariadb", "app.kubernetes.io/instance":"metal3", "app.kubernetes.io/name":"metal3-mariadb"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

[hardys]

As a workaround I was able to helm uninstall then helm install and the pv data was retained using the local-path provider - I would propose that we do not revert any changes, but instead use this issue to track adding basic upgrade coverage to CI for the metal3 chart.