fix: add sleeps between iptables and ipset cleanup

I found that without taking a brief pause between iptables cleanup and
ipset deletion, sometimes the system still thought that there were
iptables references to the ipsets and would error instead of cleaning
the ipsets.

Author: aauren

pkg/controllers/proxy/network_services_controller.go

@@ -655,6 +655,10 @@ func (nsc *NetworkServicesController) cleanupIpvsFirewall() {
 		}
 	}
 
+	// For some reason, if we go too fast into the ipset logic below it causes the system to think that the above
+	// iptables rules are still referencing the ipsets below, and we get errors
+	time.Sleep(1 * time.Second)
+
 	// Clear ipsets
 	// There are certain actions like Cleanup() actions that aren't working with full instantiations of the controller
 	// and in these instances the mutex may not be present and may not need to be present as they are operating out of a

pkg/controllers/routing/network_routes_controller.go

@@ -691,6 +691,10 @@ func (nrc *NetworkRoutingController) Cleanup() {
 		klog.V(1).Infof("Error deleting Pod egress iptables rule: %s", err.Error())
 	}
 
+	// For some reason, if we go too fast into the ipset logic below it causes the system to think that the above
+	// iptables rules are still referencing the ipsets below, and we get errors
+	time.Sleep(1 * time.Second)
+
 	// delete all ipsets created by kube-router
 	// There are certain actions like Cleanup() actions that aren't working with full instantiations of the controller
 	// and in these instances the mutex may not be present and may not need to be present as they are operating out of a