what are the necessary rights for kafkaconnect and kafkaconnector to user kafka security and kafkaconnect

[cstmgl]

Hi all,
I have been working with Kafka connect and kafka connectors for a while now.
I honestly thought I was using kafka acl security but I realized today that I was not. I now activated security in Kafka and after doing that I can't for the life of me make the Kafka Connect work.
I had problem first in the configurations and then in the joining the group.
I'm reading the authorization model for resources and I think I have everything but I still fail to connect.
https://strimzi.io/docs/operators/latest/using.html#kafka_authorization_model_for_resources

I get this in the log:
Attempt to join group failed due to fatal error: Group authorization failed.","class":"org.apache.kafka.clients.consumer.internals.AbstractCoordinator$JoinGroupResponseHandler","mdc":{}}

so I wanted to know for my user what are the permissions I need just for making kafka connect work and start?
the connectors I guess I can worry about that once the connect is working but I'm a bit lost.

So far I've created this in the user (group, transactional id and internal topics) but still nothing

---
apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaUser
metadata:
  name: dataflow-connector-user
  labels:
    app.kubernetes.io/component: messaging
    strimzi.io/cluster: strimzi-cluster
spec:
  authentication:
    type: tls
  authorization:
    acls:
      - operation: Read
        resource:
          name: connect-
          patternType: prefix
          type: group
      - operation: Describe
        resource:
          name: connect-
          patternType: prefix
          type: transactionalId
      - operation: Write
        resource:
          name: connect-
          patternType: prefix
          type: transactionalId
      - operation: Describe
        resource:
          name: com.dataflow.
          patternType: prefix
          type: group
      - operation: Read
        resource:
          name: com.dataflow.
          patternType: prefix
          type: topic
      - operation: Write
        resource:
          type: topic
          name: com.dataflow.kafkaconnect.configs
          patternType: literal
      - operation: Write
        resource:
          type: topic
          name: com.dataflow.kafkaconnect.offset
          patternType: literal
      - operation: Write
        resource:
          type: topic
          name: com.dataflow.kafkaconnect.status
          patternType: literal
      - operation: Write
        resource:
          type: topic
          name: com.dataflow.kafkaconnect.dlq
          patternType: literal
    type: simple

I guess the root of my problem is that I don't really know what I need to put in the group and transactional id I would guess.

[scholzj]

This is what I normally use for my Connect cluster:

apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaUser
metadata:
  name: my-connect-cluster
  labels:
    strimzi.io/cluster: my-cluster
spec:
  authentication:
    type: tls
  authorization:
    type: simple
    acls:
    # Kafka Connects internal topics
    - resource:
        type: group
        name: connect-cluster
      operation: Read
    - resource:
        type: topic
        name: connect-cluster-configs
      operation: Read
    - resource:
        type: topic
        name: connect-cluster-configs
      operation: Describe
    - resource:
        type: topic
        name: connect-cluster-configs
      operation: Write
    - resource:
        type: topic
        name: connect-cluster-configs
      operation: Describe
    - resource:
        type: topic
        name: connect-cluster-configs
      operation: Create
    - resource:
        type: topic
        name: connect-cluster-status
      operation: Read
    - resource:
        type: topic
        name: connect-cluster-status
      operation: Describe
    - resource:
        type: topic
        name: connect-cluster-status
      operation: Write
    - resource:
        type: topic
        name: connect-cluster-status
      operation: Describe
    - resource:
        type: topic
        name: connect-cluster-status
      operation: Create
    - resource:
        type: topic
        name: connect-cluster-offsets
      operation: Read
    - resource:
        type: topic
        name: connect-cluster-offsets
      operation: Describe
    - resource:
        type: topic
        name: connect-cluster-offsets
      operation: Write
    - resource:
        type: topic
        name: connect-cluster-offsets
      operation: Describe
    - resource:
        type: topic
        name: connect-cluster-offsets
      operation: Create

Not, this is just Connect it self - the connectors will need more, but that really depends on the connector. I do not think I ever gave it anything on transactional IDs. Not sure if this depends on some Connect configuration or belongs to some connector.

The clients (Connect just wraps the clients) normally log very general error only. But the broker logs normally log full details of what was requested. So that is normally useful for debugging any ACL issues.

[cstmgl]

thank you, I eventually managed to make it work because I was missing the access to the actual group itselft. but now it's back not working I'm a bit lost but I'm investigating the log. somehow it really complais about the internal topics like the config and offset but I believe I already gave it rights for those topics but it just refuses to work.
Anyway I guess I was trying to understand what was the transactional ID used for and see if that would be the reason but in fact it just really complains about those topics

[tombentley]

The transactional id is only needed for transactional producers and currently that doesn't really include Kafka Connect (I'm sure you could configure a transactional producer for a connector, but it wouldn't give you exactly-once semantics). KIP-618 will add support for exactly once source connections, which would then require transactional id support.

[cstmgl]

thank you both