Recreate configuration after cluster has been recreated

[bo0ts]

Currently the operator does not notice when a cluster referenced by topology objects is deleted and recreated. This leads to a situation where topology objects show up as "Ready" while they are not. It would be nice if the operator could either

• periodically (or on rabbitmqcluster events) check the state of the object and act accordingly
• watch create/delete events on a referenced rabbitmqcluster and recreate the objects

[ChunyiLyu]

Hi @bo0ts, for "periodically (or on rabbitmqcluster events) check the state of the object and act accordingly", you can do that by setting an environment variable in the topology operator pod for time based reconcliation.

For example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: messaging-topology-operator
spec:
  template:
    spec:
      containers:
      - command:
        - /manager
        env:
        - name: SYNC_PERIOD
          value: 30m

Setting it to 30 minutes means the operator will reconcile all objects it manages every 30 minutes. Without setting this envVar there is no time based reconciliation. By default reconciliation only happens when an object gets create/update/delete.

[bo0ts]

I'm aware of that feature, but that seems like a poor workaround for just having the topology operator getting notified on rabbitmq cluster changes. Why not just add a Watches(...) statement for the TopologyReconciler?

[github-actions]

This issue has been marked as stale due to 60 days of inactivity. Stale issues will be closed after a further 30 days of inactivity; please remove the stale label in order to prevent this occurring.

[github-actions]

Closing stale issue due to further inactivity.

[VannTen]

Could this be reconsidered ?
Since the topology should target a single cluster object, there should not be scalability concern to watches a related object for changes and re-trigger reconciliation.

This would also solves the limitation regarding rabbitmq.com/topology-allowed-namespaces: "my-app" needing manual intervention on the topology objects instead of automatically resolving.

Can this be re-opened or should we make a different issue ?

[Zerpet]

I understand that the preferred outcome is to add a Watch() in the topology operator to react to update/delete events on RabbitmqCluster objects. What should happen when a RabbitmqCluster object is deleted?

I'm inclined to delete cascade topology objects, because the operator doesn't know if the rabbitmq cluster is coming back. For updates, we can run the current reconcile logic.

What do you think @VannTen ?

[bo0ts]

@Zerpet Please don't remove objects without an ownership relation. The RabbitmqCluster does not own any of the topology objects and they should not be deleted when the cluster is deleted. A few other operators and their behaviors: the argocd operator does not delete any appprojects, when the argocd instance is deleted.

Rather just set a Failure condition on the status of the topology object.