Configuring Scram-sha-512 and tls authentication for kafka cluster

[Lokeshmaxi]

Suggestion / Problem

Configuring Scram-sha-512 and tls authentication for kafka cluster.
I have deployed strimzi kafka cluster in kubernetes cluster, we need to enable authentication for connecting bootstrap server from the client, so i have tried and configured SCRAM-SHA-512 authentication with authorization type "simple" and created one user with all privilege(for testing), now my question is, how to pass client properties which consist of SCRAM-SHA-512,Username and pw in the kafka cluster inside the pod, there is Vi editor inside the kafka container, and is creating client properties need to everytime when pod restarts?,is there way to pass it in within config properties of kafka, and one more help is there any detailed document to implement to authentication and authorization for all type for strimzi kafka, we are planning implement kafka for our production environment soon, have attached my configuration screenshot as reference, please help

Regards
Lokeshkumar

Documentation Link

No response

[scholzj]

I'm not entirely sure what the question is. You should never run your Kafka client from inside the broker pods. That can cause problems for example because the container might run out of memory, might degrade performance etc. When you create the KafkaUser resource, a Kubernetes Secret is created with the credentials. You can use that in your Kafka client in any way you want - and in any way the client or programming language you use needs.

[Lokeshmaxi]

post that kafkauser creation, i have executed kafka pod and tried to create topic using kafka cli, but it is throwing Authorization denied, my question is that we can also able to create topic using the kafka cli within the pod right ?

[scholzj]

You should never execute into the KAfka broker pods to run some command. Start your own pod and run it from there. In any case, if you get authorization error:

• Make sure you are authenticated (in your config, the port 9092 has no authentication, so it will likely always get "Not authorized" error)
• Check the broker log to see what exactly is the ACL right you are missing.

[Lokeshmaxi]

I have configured authorization from 9094 to 9092, and tried to create topic, but getting this error from kafka pod=listenerType=ZK_BROKER Failed authentication (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector) [data-plane-kafka-network-thread-0-ListenerName(PLAIN-9092)-SASL_PLAINTEXT-5]

[scholzj]

Well, that suggests you have something wrong with the authenication setup of your client.

…

On Tue, Nov 14, 2023, 21:10 Lokeshmaxi ***@***.***> wrote: I have configured authorization from 9094 to 9092, and tried to create topic, but getting this error from kafka pod=listenerType=ZK_BROKER Failed authentication (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector) [data-plane-kafka-network-thread-0-ListenerName(PLAIN-9092)-SASL_PLAINTEXT-5] — Reply to this email directly, view it on GitHub <#9349 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLFOR3QIZ2UWT4Y4LGUUT3YEPF4HAVCNFSM6AAAAAA7K4SX2KVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TKNRZGQ4DE> . You are receiving this because you commented.Message ID: <strimzi/strimzi-kafka-operator/repo-discussions/9349/comments/7569482@ github.com>

[Lokeshmaxi]

Hi Schokzj, can you please share the some other document to implement Scram-sha-512 for authentication for kafka, i have tried many times, still it is not working, please guide me

[Lokeshmaxi]

`

Attached` Scram-sha-512 configuration to kafka, please let me know is this correct or not

[scholzj]

This is not how you configure Strimzi. All the options around security you use there are forbidden and will not be used for anything. If you want to configure authentication, you have to do it in the .spec.kafka.listeners.authentication section of the Kafka custom resource. There are examples in the docs and on our GitHub. The same applies to authorization.