Experiments With Same Name & Template Name Doesn't Update the Replica Sets Upon Updates

[satyamsareen007]

Checklist:

• I've included steps to reproduce the bug.
• I've included the version of argo rollouts.

Describe the bug

I have a standalone experiment resource that we use to test the new version. Every time we have changes to test, we update the experiment config, but the experiment name & tempate name remain the same (which decides the replica set name).

We have observed that the experiment resource gets updated with the new changes, but the replica set that is reponsible of spinning up the experiment pods doesn't get updated. So pods get created with the old config.

To Reproduce

Create an experiment with any config --> experiment runs --> push a change to the experiment(could be image tag) --> retry experiment --> experiment pods don't have the new image tag

Expected behavior

Experiments should update their replica set if a change has been pushed to them

Version

1.8.3

Logs

Nothing useful from the logs

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

[satyamsareen007]

@krapie @zachaller for viz

[krapie]

I see that the experiment RS name does not contain pod hash unlike other RS, and experiment reconciler does not check the diff and recreate the new RS even if sees RS diff event.

argo-rollouts/experiments/replicaset.go

Lines 179 to 181 in cb1c33d

	rs := appsv1.ReplicaSet{
	ObjectMeta: metav1.ObjectMeta{
	Name: fmt.Sprintf("%s-%s", experiment.Name, template.Name),

Not sure if this is intended or not. But if this is not intended, it may need to change experiment reconciler to apply RS changes.

[satyamsareen007]

thanks @krapie, for looking into this

As a workaround, we are deleting the replica set before retrying the experiment, so that the freshly created replica set picks up the new changes. The other workaround could be to manually insert the template hash into either the experiment name or the template name so that the created replica set gets a new name and gets recreated.

It would be amazing if this could be supported within the controller logic itself.

For embedded experiments within a rollout, we don't get this problem there, as the template hash of the new replica set is automatically a part of the experiment name, thus making the experiment's replica set name unique.

Only for standalone experiments, I believe this is a problem

[krapie]

I also think this RS non-change issue on Experiment retry is not an expected behavior.
Created PR #4582 for this and it works for me.