is there a way the ClusterOperator can be restarted automatically ?

[hari819]

@scholzj ,

I am using latest vesrsion of strimzi operator and i am trying to create multiple kafka clusters by using the watchAnyNamespace=true option in strimzi-operator ,
but i am seeing this warning "Failed to acquire lock for kafka cluster lock::kafka::myproject::my-cluster within 10000ms ,

after this warning , operator goes non-responsive and no kafka clusters can be created/modified untill i restart the operator ,
i searched for charts/values to find this "10000ms" value , but i could not find ,

is there a way i can increase this timeout to stop this warning and in turn make operator work, because this may be just a warning , but the clusters are not getting created which is kind of a blocker for me ,

could you please let me know if we can restart this operator automatically based on some health checks , non-responsiveness ,or based on error "Failed to acquire lock "

Thanks,

[scholzj]

The operator has health checks, so it should restart it self. The message you listed does normally not mean it is in anyway non-responsive. It just means it is doing something else. If you think it doesn't do anything afterwards, please provide a full log from the operator, ideally on the DEBUG level (but even without it if you have it without it).

[hari819]

@scholzj , could you please let me know how much time does it take to again start creating/modifying cluster after we see the warning, "Failed to acquire lock " , I waited for some time after this warning , in the end had to kill the pod manually ,

the health checks are not helping in this case ,
i did not do any customization for the operator , i used the default one that comes with the helm charts ,
I will modify the log level to debug and try to capture the full log ,

Thanks,

[scholzj]

The warning means there is another reconilication in progress. The timeout for that is by default 5 minutes.