Unable to connect to kafka-cluster

[shiva-appani-hash]

As suggested created cluster using listener authentication type as SCRAM-SHA-512 instead of PLAIN, client connections are not successful.

apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata: name: kafka spec: kafka: version: 3.7.0 replicas: 3 listeners: - name: plain port: 9092 type: internal tls: false authentication: type: scram-sha-512 - name: tls port: 9093 type: internal tls: true authentication: type: scram-sha-512 - name: external port: 9094 type: loadbalancer tls: true authentication: type: scram-sha-512 config: offsets.topic.replication.factor: 3 transaction.state.log.replication.factor: 3 transaction.state.log.min.isr: 1 log.message.format.version: "3.7" inter.broker.protocol.version: "3.7" storage: type: persistent-claim size: 50Gi zookeeper: replicas: 3 storage: type: persistent-claim size: 50Gi entityOperator: topicOperator: {} userOperator: {}

kafka-user manifest:

apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaUser metadata: name: kafka-user labels: strimzi.io/cluster: kafka spec: authentication: type: scram-sha-512 authorization: type: simple acls: - resource: type: topic name: '*' patternType: literal operation: All - resource: type: group name: '*' patternType: literal operation: All

Below is our kafka-client config:

kafka-console-producer.sh
--broker-list kafka-kafka-brokers:9092
--topic test
--producer.config <(echo -e "security.protocol=SASL_SSL\nsasl.mechanism=SCRAM-SHA-512\nsasl.username=kafka-user\nsasl.password=REDACTED"

Ref: #10108

[shiva-appani-hash]

Seeing below exception while connecting to brokers from client pod.
@scholzj Do we have need to include keystore and truststore location in client-config.
Could you please help us with the documentation if any.

I have no name!@kafka-client:/$ kafka-console-producer.sh \
>   --broker-list kafka-kafka-brokers:9092 \
>   --topic test \
>   --producer.config <(echo -e "security.protocol=SASL_SSL\nsasl.mechanism=SCRAM-SHA-512\nsasl.username=kafka-user\nsasl.password=REDACTED")
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:473)
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:294)
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:321)
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:306)
        at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:98)
        at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Caused by: org.apache.kafka.common.KafkaException: Failed to create new NetworkClient
        at org.apache.kafka.clients.ClientUtils.createNetworkClient(ClientUtils.java:252)
        at org.apache.kafka.clients.ClientUtils.createNetworkClient(ClientUtils.java:162)
        at org.apache.kafka.clients.producer.KafkaProducer.newSender(KafkaProducer.java:523)
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:462)
        ... 5 more
Caused by: java.lang.SecurityException: java.io.IOException: /opt/kafka/client-ssl.properties (No such file or directory)
        at java.base/sun.security.provider.ConfigFile$Spi.<init>(ConfigFile.java:139)
        at java.base/sun.security.provider.ConfigFile.<init>(ConfigFile.java:104)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128)
        at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:347)
        at java.base/java.lang.Class.newInstance(Class.java:645)
        at java.base/javax.security.auth.login.Configuration$2.run(Configuration.java:258)
        at java.base/javax.security.auth.login.Configuration$2.run(Configuration.java:249)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
        at java.base/javax.security.auth.login.Configuration.getConfiguration(Configuration.java:248)
        at org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
        at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:103)
        at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:87)
        at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:167)
        at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
        at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:119)
        at org.apache.kafka.clients.ClientUtils.createNetworkClient(ClientUtils.java:223)
        ... 8 more
Caused by: java.io.IOException: /opt/kafka/client-ssl.properties (No such file or directory)
        at java.base/sun.security.provider.ConfigFile$Spi.ioException(ConfigFile.java:669)
        at java.base/sun.security.provider.ConfigFile$Spi.init(ConfigFile.java:265)
        at java.base/sun.security.provider.ConfigFile$Spi.<init>(ConfigFile.java:137)
        ... 27 more
I have no name!@kafka-client:/$

[scholzj]

I'm not familiar with the way you are trying to use the client. You should probably check Kafka docs and/or the console consumer help to see how to configure it. You should also read the error before you share it and ask for help as it quite clearly suggests that you misconfigured it somehow.