Help to understand how scram-sha-512 auth works

[ginwakeup]

Hey,

Following my question the other day, I've now setup scram-sha-512 authentication on the kafka cluster external listeners:

    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: false
      - name: external
        port: 9094
        type: loadbalancer
        tls: false
        authentication:
          type: scram-sha-512

Then I created an user with same auth type:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: user
  labels:
    strimzi.io/cluster: messaging
spec:
  authentication:
    type: scram-sha-512

And I am using it to connect from a python consumer:

from kafka import KafkaConsumer


sasl_mechanism = 'SCRAM-SHA-512'

# To consume latest messages and auto-commit offsets
consumer = KafkaConsumer('test',
                         group_id='test',
                         auto_offset_reset='earliest',
                         bootstrap_servers=['******'],
                         sasl_plain_username="user",
                         sasl_mechanism=sasl_mechanism,
                         security_protocol="SASL_SSL",
                         sasl_plain_password="******"
                         )
...

Kafka gives me an error on the handshake:

2023-02-07 22:32:36,755 INFO [SocketServer listenerType=ZK_BROKER, nodeId=1] Failed authentication with /1****128 (channelId=1****.16:9094-1*****.128:45471-7) (Failing SASL authentication due to invalid receive size) (org.apache.kafka.common.network.Selector) [data-plane-kafka-network-thread-1-ListenerName(EXTERNAL-9094)-SASL_PLAINTEXT-16]

I am trying to understand few things here to properly configure it:

• Do I have to use SSL even if the auth type is set to scram-sha-512? Do I need to provide certificates from the python client when I connect to the listeners?
• If so, what certificates can I use that are generated from the operator? <cluster>-clients-ca?
• I've tried to set tls: true on the kafka-cluster.yaml, but still I am not sure I understand what certificates then I have to pass from the clients, if any at all.

Thanks

[scholzj]

If you set tls: false on the listener, you are not using encryption. In that case, your security.protocol in the client has to be SASL_PLAINTEXT and not SASL_SSL. That is likely causing the issue you see.

You do not need to use TLS encryption with SASL - SASL works fine without it and SCRAM-SHA-512 does not send raw passwords over the network. But obviously the messages you are sending will not be encrypted. So normally, encryption will be recommended ... but it is up to you to decide. You just need to configure the client accordingly.

If you enable the TLS, with the configuration you have for the listener, you should get the Cluster CA public key from the secret named <cluster-name>-cluster-ca-cert and use it in your client. I do not use Python, so I'm afraid I cannot give you the exact option for the Python client, but I'm sure you will be able to find that somewhere.

[ginwakeup]

@scholzj Thank you very much. For now I will set it to SASL_PLAINTEXT and it works fine, but I'll look more into enabling TLS.

I tried to use the <cluster-name>-cluster-ca-cert after setting tsl:true, but it still didn't let me login, giving me an error message like:

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1045)

I'll have to look more into it on the python side probably.

[scholzj]

Well, the certificate is a self-signed Root CA essentially. But that is not a reason to fail on its own. As I said, not a Python user I'm afraid. So not sure if this is something Python-specific or what it means here. Sorry.

[ginwakeup]

Well, the certificate is a self-signed Root CA essentially. But that is not a reason to fail on its own. As I said, not a Python user I'm afraid. So not sure if this is something Python-specific or what it means here. Sorry.

Nothing to be sorry about, your solution was more than enough for now. Thank you again.