Produce message for TLS & authentication SASL/SCRAM & authorization

[chary1112004]

Hi,

I would like to enable TLS between client and broker. Combine with using authentication SASL/SCRAM and authorization.

kafka-ephemeral.yaml

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 3.4.0
    replicas: 3
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
        authentication:
          type: scram-sha-512
      - name: tls
        port: 9093
        type: internal
        tls: true
        authentication:
          type: scram-sha-512
    authorization:
      type: simple
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      default.replication.factor: 3
      min.insync.replicas: 2
      inter.broker.protocol.version: "3.4"
    storage:
      type: ephemeral
  zookeeper:
    replicas: 3
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}

kafka user

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: my-user
  labels:
    strimzi.io/cluster: my-cluster
spec:
  authentication:
    type: scram-sha-512
  authorization:
    type: simple
    acls:
      # Example ACL rules for consuming from my-topic using consumer group my-group
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operations:
          - Describe
          - Read
        host: "*"
      - resource:
          type: group
          name: "*"
          patternType: literal
        operations:
          - All
        host: "*"
      # Example ACL rules for producing to topic my-topic
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operations:
          - Create
          - Describe
          - Write
        host: "*"

Create kafka pod with image quay.io/strimzi/kafka:0.34.0-kafka-3.4.0
Inside kafka pod, I try to produce message as below config:

bash-4.4$ ls -la /usr/lib/jvm/jre/lib/security/cacerts
lrwxrwxrwx 1 root root 21 Jan 14 03:52 /usr/lib/jvm/jre/lib/security/cacerts -> /etc/pki/java/cacerts

bash-4.4$ cp /usr/lib/jvm/jre/lib/security/cacerts /tmp/kafka.client.truststore.jks

bash-4.4$ cat /tmp/client_sasl.properties
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
ssl.truststore.location=/tmp/kafka.client.truststore.jks
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="my-user" password="<pwd>";

bash-4.4$ bin/kafka-console-producer.sh --broker-list my-cluster-kafka-bootstrap:9093 --producer.config /tmp/client_sasl.properties --topic my-topic

[2023-04-14 09:35:20,757] ERROR [Producer clientId=console-producer] Connection to node -1 (my-cluster-kafka-bootstrap/172.20.28.225:9093) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)

It raise error SSL handshake failed

Any solution to resolve this issue would be very much appreciated.

Thanks!

[scholzj]

I think your trust store is not correct. You have to use the certificate from the my-cluster-cluster-ca-cert secret. Either take the ca.crt file and create your own trust store or use the ca.p12 file which Java can consume directly (in that case you would also need to set the password to the password which is in the secret).

[chary1112004]

I think your trust store is not correct. You have to use the certificate from the my-cluster-cluster-ca-cert secret. Either take the ca.crt file and create your own trust store or use the ca.p12 file which Java can consume directly (in that case you would also need to set the password to the password which is in the secret).

Thanks @scholzj, yes, you are right. The case I posted above that when I configure with kafka in aws so it may be little bit different with strimzi kafka.
I have done following then it works:

trust store

kubectl get secret my-cluster-cluster-ca-cert -n kafka -o jsonpath='{.data.ca\.p12}' | base64 -d > my-cluster-ca.p12

password trust store

kubectl get secret my-cluster-cluster-ca-cert -n kafka -o jsonpath='{.data.ca\.password}' | base64 -d

/tmp/client_sasl.properties

security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
ssl.truststore.location=/tmp/my-cluster-ca.p12
ssl.truststore.password=<password trust store>
ssl.truststore.type=PKCS12
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="my-user" password="<pwd>";

Then we could close this one!

Thanks!