Cannot create kafka cluster

[darxsideblues]

Hi!

I've stumbled upon some annoying error lately, when I'm trying to install strimzi kafka cluster I got a errors like that:

Status: Conditions: Last Transition Time: 2022-01-26T07:20:49.706Z Message: Invalid DNS name: franz-zookeeper-client.kafka.svc.cluster.local kubernetes.default.strimzi.svc.cluster.local Reason: IllegalArgumentException Status: True Type: NotReady Observed Generation: 1 Events: <none>

Sometimes this error doesn't occur, sometimes it is gone after I delete namespace completely and reinstall from scratch.

It is pretty very annoying, because of it's unpredictable nature.

Sorry I'm new here, so maybe it's something ridiculous, but however I'm struggling with these for a while.

Any assistance you can provide would be greatly appreciated, thanks!

[scholzj]

That seems like some DNS issue when resolving some address. Can you share the full Cluster Operator log when it happens?

[darxsideblues]

Yes, here they are:

2022-01-26 07:41:31 INFO  ClusterOperator:118 - Triggering periodic reconciliation for namespace strimzi
2022-01-26 07:41:31 INFO  ClusterOperator:118 - Triggering periodic reconciliation for namespace kafka
2022-01-26 07:41:31 INFO  AbstractOperator:219 - Reconciliation #23(timer) Kafka(kafka/franz): Kafka franz will be checked for creation or modification
2022-01-26 07:41:31 ERROR AbstractOperator:240 - Reconciliation #23(timer) Kafka(kafka/franz): createOrUpdate failed
java.lang.IllegalArgumentException: Invalid DNS name: franz-zookeeper-client.kafka.svc.cluster.local kubernetes.default.strimzi.svc.cluster.local
	at io.strimzi.certs.Subject$Builder.addDnsName(Subject.java:47) ~[io.strimzi.certificate-manager-0.25.0.jar:0.25.0]
	at io.strimzi.operator.cluster.model.ClusterCa.lambda$generateZkCerts$0(ClusterCa.java:132) ~[io.strimzi.cluster-operator-0.25.0.jar:0.25.0]
	at io.strimzi.operator.cluster.model.Ca.maybeCopyOrGenerateCerts(Ca.java:422) ~[io.strimzi.operator-common-0.25.0.jar:0.25.0]
	at io.strimzi.operator.cluster.model.ClusterCa.generateZkCerts(ClusterCa.java:143) ~[io.strimzi.cluster-operator-0.25.0.jar:0.25.0]
	at io.strimzi.operator.cluster.model.ZookeeperCluster.generateCertificates(ZookeeperCluster.java:486) ~[io.strimzi.cluster-operator-0.25.0.jar:0.25.0]
	at io.strimzi.operator.cluster.operator.assembly.KafkaAssemblyOperator$ReconciliationState.lambda$zkGenerateCertificates$54(KafkaAssemblyOperator.java:1441) ~[io.strimzi.cluster-operator-0.25.0.jar:0.25.0]
	at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:160) ~[io.vertx.vertx-core-4.1.2.jar:4.1.2]
	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:96) ~[io.vertx.vertx-core-4.1.2.jar:4.1.2]
	at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:158) ~[io.vertx.vertx-core-4.1.2.jar:4.1.2]
	at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76) ~[io.vertx.vertx-core-4.1.2.jar:4.1.2]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty.netty-common-4.1.66.Final.jar:4.1.66.Final]
	at java.lang.Thread.run(Thread.java:829) [?:?]
2022-01-26 07:41:31 WARN  AbstractOperator:481 - Reconciliation #23(timer) Kafka(kafka/franz): Failed to reconcile
java.lang.IllegalArgumentException: Invalid DNS name: franz-zookeeper-client.kafka.svc.cluster.local kubernetes.default.strimzi.svc.cluster.local
	at io.strimzi.certs.Subject$Builder.addDnsName(Subject.java:47) ~[io.strimzi.certificate-manager-0.25.0.jar:0.25.0]
	at io.strimzi.operator.cluster.model.ClusterCa.lambda$generateZkCerts$0(ClusterCa.java:132) ~[io.strimzi.cluster-operator-0.25.0.jar:0.25.0]
	at io.strimzi.operator.cluster.model.Ca.maybeCopyOrGenerateCerts(Ca.java:422) ~[io.strimzi.operator-common-0.25.0.jar:0.25.0]
	at io.strimzi.operator.cluster.model.ClusterCa.generateZkCerts(ClusterCa.java:143) ~[io.strimzi.cluster-operator-0.25.0.jar:0.25.0]
	at io.strimzi.operator.cluster.model.ZookeeperCluster.generateCertificates(ZookeeperCluster.java:486) ~[io.strimzi.cluster-operator-0.25.0.jar:0.25.0]
	at io.strimzi.operator.cluster.operator.assembly.KafkaAssemblyOperator$ReconciliationState.lambda$zkGenerateCertificates$54(KafkaAssemblyOperator.java:1441) ~[io.strimzi.cluster-operator-0.25.0.jar:0.25.0]
	at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:160) ~[io.vertx.vertx-core-4.1.2.jar:4.1.2]
	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:96) ~[io.vertx.vertx-core-4.1.2.jar:4.1.2]
	at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:158) ~[io.vertx.vertx-core-4.1.2.jar:4.1.2]
	at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76) ~[io.vertx.vertx-core-4.1.2.jar:4.1.2]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty.netty-common-4.1.66.Final.jar:4.1.66.Final]
	at java.lang.Thread.run(Thread.java:829) [?:?]

[scholzj]

Well, sharing the full log would have been more helpful. You might be riunning into #5444 => you can try to set the DNS suffix of your Kiube cluster manually using the environment variable KUBERNETES_SERVICE_DNS_DOMAIN in the Cluster Operator deployment.

[darxsideblues]

Wow, it seems like it helped! Thanks a lot!