How interal producer with port 9092 access an topic that has authorization

[barryzhounb]

First, define a Kafka cluster named dlake-cluster, 9094 for external access with SSL protocol, and 9092 with plain protocol for internal access

apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
  name: dlake-cluster
  namespace: abc
spec:
  kafka:
    authorization:
      type: simple
    listeners:
      external:
        authentication:
          type: tls
        port: 9094
        tls: true
        type: route
      plain:
        port: 9092
        tls: false
        type: internal
    replicas: 3
    storage:
      type: ephemeral
  zookeeper:
    replicas: 3
    storage:
      type: ephemeral

Then, define a topic named dlake-topic

apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaTopic
metadata:
  name: dlake-topic
  namespace: abc
  labels:
    strimzi.io/cluster: dlake-cluster
spec:
  config:
    retention.ms: 604800000
    segment.bytes: 1073741824
  partitions: 6
  replicas: 3

Then, define a user ACL named dlake-user

piVersion: kafka.strimzi.io/v1beta1
kind: KafkaUser
metadata:
  name: dlake-user
  namespace: abc
  labels:
    strimzi.io/cluster: dlake-cluster
spec:
  authentication:
    type: tls
  authorization:
    acls:
      - host: '*'
        operation: All
        resource:
          name: dlake-topic
          patternType: literal
          type: topic
        type: allow
      - host: '*'
        operation: All
        resource:
          name: dlake-group
          patternType: literal
          type: group
        type: allow
      - host: '*'
        operation: All
        resource:
          name: dlake-cluster
          patternType: literal
          type: cluster
        type: allow
    type: simple

Now I retrieve cluster-ca-cert, dlake-user certificate and keystore, then generated truststore.jks and keystore.jks, and generate file ssl-config for producer/consumer client as following.

security.protocol=SSL
ssl.trustmanager.algorithm=PKIX
ssl.truststore.location=./truststore.jks
ssl.truststore.password=123456
ssl.truststore.type=JKS
ssl.keystore.location=./keystore.jks
ssl.keystore.password=123456
ssl.keystore.type=JKS

Then external client (producer/consumer) use the above configuration, they can access dlake-topic via Routes port 443, it did authentication and authorization with SSL protocol succesfully.

Now, an app inside internal cluster want to access dlake-topic via port 9092. As you know, port 9092 listener doesn't need to do authentication, but app want to access dlake-topic, in fact, in order to access dlake-topic, it need some permission with authorization for dlake-topic. I am confused here and get stuck here. How this app inside OpenShift access dlake-topic?

(1) Does app inside internal cluster need to use the same ssl-config?
(2) is it possible for app inside internal cluster to access dlake-topic without athentication and authoriation?
(3) Could you please give a complete solution how app inside internal cluster access dlake-topic?

[scholzj]

Is this the same question as answered on https://stackoverflow.com/questions/66510182/topicauthorizationexception? In any case the answer there seems to apply!

[barryzhounb]

Hi @scholzj , if port 9092 apply no authentication to connect cluster, but topic has authorization, you mean it will be an ANONYMOUS user to access. Now how can I add permission for ANONYMOUS to access dlake-topic? Is the following correct?

piVersion: kafka.strimzi.io/v1beta1
kind: KafkaUser
metadata:
  name: ANONYMOUS
  namespace: abc
  labels:
    strimzi.io/cluster: dlake-cluster
spec:
  authentication:
    type: tls
  authorization:
    acls:
      - host: '*'
        operation: All
        resource:
          name: dlake-topic
          patternType: literal
          type: topic
        type: allow
      - host: '*'
        operation: All
        resource:
          name: dlake-group
          patternType: literal
          type: group
        type: allow
      - host: '*'
        operation: All
        resource:
          name: dlake-cluster
          patternType: literal
          type: cluster
        type: allow
    type: simple

[scholzj]

No, it is not correct. As I said in the SO response ... you have to use the Kafka Admin API or the Kafka tools to set the ACLs for the ANONYMOUS user.

[barryzhounb]

Thanks your reply. I try to find some information to do that, but I didn't find proper way. Could you please show an example how to use Kafka Admin API or the Kafka tools to set ANONYMOUS permission? Thanks

[scholzj]

I do not have an example. But there are JavaDoc on the Kafka website? http://kafka.apache.org/27/javadoc/index.html?org/apache/kafka/clients/admin/Admin.html

[barryzhounb]

Thanks @scholzj