Kafka still failing authentication on listener setup with no authentication

[ginwakeup]

Hi there,

I am trying to setup a Strimzi Kafka in a Kubernetes Cluster with 3 listeners:

• two internal listener for other kubernetes apps, with no authentication
• one external listener for external applications, which uses SCRAM_512 auth.

I am using Strimzi 0.33.0, the following is the cluster deployment:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: messaging
spec:
  kafka:
    version: 3.2.3 #Apache Kafka version
    replicas: 4
    resources:
      requests:
        cpu: 1000m
        memory: 3Gi
      limits:
        cpu: 2000m
        memory: 6Gi
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: false
      - name: external
        port: 9094
        type: loadbalancer
        tls: true
        authentication:
          type: scram-sha-512
        configuration:
          bootstrap:
            loadBalancerIP: ***
          brokers:
          - broker: 0
            loadBalancerIP: ***
          - broker: 1
            loadBalancerIP: ***
          - broker: 2
            loadBalancerIP: ***
          - broker: 3
            loadBalancerIP: ***
    authorization:
      type: simple
    config:
      auto.create.topics.enable: "true"
      offsets.topic.replication.factor: 4
      transaction.state.log.replication.factor: 4
      transaction.state.log.min.isr: 2
      default.replication.factor: 4
      min.insync.replicas: 2
      inter.broker.protocol.version: "3.2.3"
      num.recovery.threads.per.data.dir: 12
      num.network.threads: 4
      num.io.threads: 12
    readinessProbe:
      initialDelaySeconds: 30
      timeoutSeconds: 30
    storage:
      type: jbod
      volumes:
      - id: 0
        type: persistent-claim
        size: 100Gi
        deleteClaim: false
  zookeeper:
    replicas: 4
    storage:
      type: persistent-claim
      size: 100Gi
      deleteClaim: false
  entityOperator:
    topicOperator: {}
    userOperator: {}

But when I try to connect to Kafka from inside Kubernetes, I still get an authentication error when connecting on the listener using port 9092.

org.apache.kafka.common.errors.ClusterAuthorizationException: Cluster authorization failed

The exception is raised by the app kafka-ui, where I am simply connecting to the 9092 listener without specifying anything:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka-ui
  labels:
    app: kafka-ui
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: kafka-ui
      app.kubernetes.io/component: kafka-ui
  template:
    metadata:
      labels:
        app.kubernetes.io/name: kafka-ui
        app.kubernetes.io/component: kafka-ui
    spec:
      containers:
      - name: kafka-ui
        image: provectuslabs/kafka-ui:latest
        imagePullPolicy: "Always"
        env:
          - name: KAFKA_CLUSTERS_0_NAME
            value: "TechArt Kafka Cluster"
          - name: KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS
            value: "messaging-kafka-brokers:9092"
          - name: KAFKA_CLUSTERS_0_ZOOKEEPER
            value: "messaging-zookeeper-nodes:2181"
        ports:
        - containerPort: 8080
        resources:
          limits:
            memory: "2G"
            cpu: "1000m"

Reading the error, it seems that the issue is not in the authentication, but on the authorization.
But shouldn't the listener on 9092 not run any authorization check?

Am I missing something?

Thanks

[scholzj]

Please do not open bugs for something that is just a configuration issue!

Why do you think it is an authentication issue? The error clearly states that authorization failed. You have authorization enabled in your cluster and that applies to all listeners regardless of whether you enabled authentication there or now. If you don't enable authentication, any user will be able to connect and will be treated as user ANONYMOUS.

To get this working with your setup you have 3 options:

• You can disable the authorization
• You can configure authentication on all listeners
• You can make the user ANONYMOUS a super user and that way, connections going through the listeners without authentication will be allowed to do everything.

PS: Strimzi does not allow direct access to ZooKeeper. So I don't think your UI will be able to connect to messaging-zookeeper-nodes:2181

[ginwakeup]

Hi @scholzj and thank you for your reply. In the end of my message I was indeed asking regarding the authorization.
https://strimzi.io/docs/operators/latest/full/configuring.html#con-securing-kafka-authentication-str

Disabling the auth for now will do the trick. Thank you