The client cannot connect to Kafka cluster

[dockerxu123]

hello :

1.I have disabled 9094 SSL. The server still needs to provide certificate authentication. How can I solve this problem.
2. I want to enable 9094 SSL, How should I configure it? kafka-cluster.yaml

kafka-cluster.yaml
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: kafka-cluster
spec:
kafka:
version: 2.8.1
replicas: 3
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
- name: external
port: 9094
type: nodeport
tls: false
configuration:
bootstrap:
nodePort: 32410
brokers:
- broker: 0
nodePort: 32420
- broker: 1
nodePort: 32421
- broker: 2
nodePort: 32422
config:
offsets.topic.replication.factor: 3
transaction.state.log.replication.factor: 3
transaction.state.log.min.isr: 2
log.message.format.version: "2.8"
inter.broker.protocol.version: "2.8"
storage:
type: jbod
volumes:
- id: 0
type: persistent-claim
size: 10Gi
deleteClaim: false
class: rook-rtshield-block
zookeeper:
replicas: 3
storage:
type: persistent-claim
size: 10Gi
deleteClaim: false
class: rook-rtshield-block
entityOperator:
topicOperator: {}
userOperator: {}

[xiaob666]

配置错误吧，bootstrap servers 应该是你上面 kafka-cluster.yaml 中的 32410，不是 broker 地址

[dockerxu123]

换成32410也是一样报错，应该是要证书连接，但是应该需要证书的password我在集群中找不到，

[dockerxu123]

就是这个密码 ，zookeeper.ssl.keystore.password=[hidden] 我找不到

[scholzj]

Sorry, I'm not sure I follow what exactly are you trying to do. The tls flag in the listener configuration enabled or disables TLS. You seem to have it as tls: false now, so TLS is disabled. If you set it to tls: true, it will roll the brokers and enabled TLS (with node ports, you will need to disabale hostname verification to connect to it).

[scholzj]

Duplicate to #6357?

[dockerxu123]

now is tls: false ，but Offset Explorer 2.0 client cannot connect to Kafka cluster

[scholzj]

I'm afraid I have no idea what Offset Explorer is and how it is configured. The error from the Kafka broker you shared in the first post would suggest that it might be using TLS to connect to the broker with TLS disabled. But I have no idea how to disable TLS in the tool.

[scholzj]

If you would decide to enable the TLS in the broker and use it, the <cluster-name>-cluster-ca-cert secret should have a ca.p12 file to be used as a truststore (the password for it will be in the same secret). So you can use that. (and disable the hostname verification => there seems to be a checkbox for it in one of your screenshots)

[dockerxu123]

hello :  1.cluster cannot be created without CA certificate kafka.yaml apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata:   name: best-kafka-cluster spec:   kafka:     version: 2.8.0     replicas: 3     listeners:       - name: auth         port: 9095         type: internal         tls: true         authentication:           type: tls       - name: internal         port: 9092         type: internal         tls: true       - name: tls         port: 9093         type: nodeport         tls: true       - name: external         port: 9094         type: nodeport         tls: false     config:       auto.create.topics.enable: 'true'       offsets.topic.replication.factor: 3       transaction.state.log.replication.factor: 3       transaction.state.log.min.isr: 2       log.message.format.version: 2.8       inter.broker.protocol.version: 2.8       ssl.endpoint.identification.algorithm: ''     storage:       type: jbod       volumes:         - id: 0           type: persistent-claim           size: 10Gi           class: rook-rtshield-block           deleteClaim: false   zookeeper:     replicas: 3     storage:       type: persistent-claim       size: 10Gi       class: rook-rtshield-block       deleteClaim: false     logging:       type: inline       loggers:         zookeeper.root.logger: 'INFO'   entityOperator:     topicOperator: {}     userOperator: {}   clusterCa:     generateCertificateAuthority: false   clientsCa:     generateCertificateAuthority: false

…

------------------ 原始邮件 ------------------ 发件人: "strimzi/strimzi-kafka-operator" ***@***.***>; 发送时间: 2022年2月14日(星期一) 凌晨1:57 ***@***.***>; ***@***.******@***.***>; 主题: Re: [strimzi/strimzi-kafka-operator] The client cannot connect to Kafka cluster (Discussion #6343) Quick question I have SSL handshake problem to connect port 9093 which have tls-encryption enable cluster.zip as my yaml for kubectl apply -f cluster.yaml as 9093 type is Nodeport so I already add ssl.endpoint.identification.algorithm: '' in cluster.yaml but it still doesnt work here is logs of cluster-opearator Like it ignore that ssl.endpoint.identification.algorithm: '' before this I already try both use CA fron strimzi and create my own CA with your instructions https://github.com/scholzj/strimzi-custom-ca-test — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: ***@***.***>

[scholzj]

Sorry, that YAML seems to have some weird HTML formatting.

[dockerxu123]

apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata:   name: kafka-cluster spec:   kafka:     version: 2.8.1     replicas: 3     listeners:       - name: auth         port: 9095         type: internal         tls: true         authentication:           type: tls       - name: internal         port: 9092         type: internal         tls: true       - name: tls         port: 9093         type: nodeport         tls: true       - name: external         port: 9094         type: nodeport         tls: true         configuration:           bootstrap:             nodePort: 32410           brokers:            - broker: 0              nodePort: 32420            - broker: 1              nodePort: 32421            - broker: 2              nodePort: 32422     config:       offsets.topic.replication.factor: 3       transaction.state.log.replication.factor: 3       transaction.state.log.min.isr: 2       log.message.format.version: "2.8"       inter.broker.protocol.version: "2.8"       ssl.endpoint.identification.algorithm: ''     storage:       type: jbod       volumes:       - id: 0         type: persistent-claim         size: 10Gi         deleteClaim: false         class: rook-rtshield-block   zookeeper:     replicas: 3     storage:       type: persistent-claim       size: 10Gi       deleteClaim: false       class: rook-rtshield-block   entityOperator:     topicOperator: {}     userOperator: {} 1. external SSL  enable 2.Where is the keystore password saved? Is it my ca.password above，

…

------------------ 原始邮件 ------------------ 发件人: "strimzi/strimzi-kafka-operator" ***@***.***>; 发送时间: 2022年2月14日(星期一) 中午11:01 ***@***.***>; ***@***.******@***.***>; 主题: Re: [strimzi/strimzi-kafka-operator] The client cannot connect to Kafka cluster (Discussion #6343) Sorry, that YAML seems to have some weird HTML formatting. — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: ***@***.***>

[dockerxu123]

1.Where is the password saved? I need this password to connect to the Kafka cluster

…

------------------ 原始邮件 ------------------ 发件人: "strimzi/strimzi-kafka-operator" ***@***.***>; 发送时间: 2022年2月14日(星期一) 中午11:01 ***@***.***>; ***@***.******@***.***>; 主题: Re: [strimzi/strimzi-kafka-operator] The client cannot connect to Kafka cluster (Discussion #6343) Sorry, that YAML seems to have some weird HTML formatting. — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: ***@***.***>

[dockerxu123]

kafka:     Offset Explorer 2.0           The download link is as follows https://www.kafkatool.com/download.html

…

------------------ 原始邮件 ------------------ 发件人: "strimzi/strimzi-kafka-operator" ***@***.***>; 发送时间: 2022年2月14日(星期一) 中午11:01 ***@***.***>; ***@***.******@***.***>; 主题: Re: [strimzi/strimzi-kafka-operator] The client cannot connect to Kafka cluster (Discussion #6343) Sorry, that YAML seems to have some weird HTML formatting. — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: ***@***.***>

[dockerxu123]

hello Jakub Scholz:  请教你查看问题，我用Offset Explorer 2.0客户端连接创建test topic显示如下日志 Principal = User:ANONYMOUS is Denied Operation = Describe from host = 192.168.5.137 我连接的用户名是my-user 为什么kafka日志显示ANONYMOUS，我什么地方存在问题？ 我想实现的需求kafka外部 连接external 9094通过 authentication:  type: scram-sha-512连接至kafka集群，并且可以创建topic，生产者及消费者。求助！ kafka.yaml文件如下： apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata:   name: my-cluster spec:   kafka:     version: 3.0.0     replicas: 3     listeners:       - name: plain         port: 9092         type: internal         tls: true         configuration:           useServiceDnsDomain: true       - name: tls         port: 9093         type: internal         tls: true         authentication:           type: scram-sha-512       - name: external         port: 9094         type: nodeport         tls: false #        authentication: #          type: scram-sha-512         configuration:           bootstrap:             nodePort: 32410           brokers:            - broker: 0              nodePort: 32420            - broker: 1              nodePort: 32421            - broker: 2              nodePort: 32422     authorization:       type: simple       superUsers:         - my-user     config:       offsets.topic.replication.factor: 3       transaction.state.log.replication.factor: 3       transaction.state.log.min.isr: 2       default.replication.factor: 3       min.insync.replicas: 2       inter.broker.protocol.version: "3.0"     storage:       type: jbod       volumes:       - id: 0         type: persistent-claim         size: 10Gi         deleteClaim: false         class: rook-rtshield-block   zookeeper:     replicas: 3     storage:       type: persistent-claim       size: 10Gi       deleteClaim: false       class: rook-rtshield-block   entityOperator:     topicOperator: {}     userOperator: {} topic.yaml文件如下： apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaTopic metadata:   name: my-topic   labels:     strimzi.io/cluster: my-cluster spec:   partitions: 10   replicas: 3   config:     retention.ms: 7200000     segment.bytes: 1073741824 kafkauser.yaml文件如下： apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaUser metadata:   name: my-user   labels:     strimzi.io/cluster: my-cluster spec:   authentication:     type: scram-sha-512   authorization:     type: simple     acls:       # Example ACL rules for consuming from my-topic using consumer group my-group       - resource:           type: topic           name: my-topic           patternType: literal         operation: Read         host: "*"       - resource:           type: topic           name: my-topic           patternType: literal         operation: Describe         host: "*"       - resource:           type: group           name: my-group           patternType: literal         operation: Read         host: "*"       # Example ACL rules for producing to topic my-topic       - resource:           type: topic           name: my-topic           patternType: literal         operation: Write         host: "*"       - resource:           type: topic           name: my-topic           patternType: literal         operation: Create         host: "*"       - resource:           type: topic           name: my-topic           patternType: literal         operation: Describe         host: "*"

[dockerxu123]

hello Jakub Scholz:  请教你查看问题，我用Offset Explorer 2.0客户端连接创建test topic显示如下日志 Principal = User:ANONYMOUS is Denied Operation = Describe from host = 192.168.5.137 我连接的用户名是my-user 为什么kafka日志显示ANONYMOUS，我什么地方存在问题？ 我想实现的需求kafka外部 连接external 9094通过 authentication:  type: scram-sha-512连接至kafka集群，并且可以创建topic，生产者及消费者。求助！ Please check the problem. I use the offset Explorer 2.0 client connection to create a test topic. The following log is displayed Principal = User:ANONYMOUS is Denied Operation = Describe from host = 192.168.5.137 My connected user name is my user. Why does the Kafka log show anonymous? Where do I have problems? I want to realize the requirement that Kafka external 9094 connects to Kafka cluster through authentication: Type: scram-sha-512, and can create topics, producers and consumers. seek help! kafka.yaml文件如下： apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata:   name: my-cluster spec:   kafka:     version: 3.0.0     replicas: 3     listeners:       - name: plain         port: 9092         type: internal         tls: true         configuration:           useServiceDnsDomain: true       - name: tls         port: 9093         type: internal         tls: true         authentication:           type: scram-sha-512       - name: external         port: 9094         type: nodeport         tls: false #        authentication: #          type: scram-sha-512         configuration:           bootstrap:             nodePort: 32410           brokers:            - broker: 0              nodePort: 32420            - broker: 1              nodePort: 32421            - broker: 2              nodePort: 32422     authorization:       type: simple       superUsers:         - my-user     config:       offsets.topic.replication.factor: 3       transaction.state.log.replication.factor: 3       transaction.state.log.min.isr: 2       default.replication.factor: 3       min.insync.replicas: 2       inter.broker.protocol.version: "3.0"     storage:       type: jbod       volumes:       - id: 0         type: persistent-claim         size: 10Gi         deleteClaim: false         class: rook-rtshield-block   zookeeper:     replicas: 3     storage:       type: persistent-claim       size: 10Gi       deleteClaim: false       class: rook-rtshield-block   entityOperator:     topicOperator: {}     userOperator: {} topic.yaml文件如下： apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaTopic metadata:   name: my-topic   labels:     strimzi.io/cluster: my-cluster spec:   partitions: 10   replicas: 3   config:     retention.ms: 7200000     segment.bytes: 1073741824 kafkauser.yaml文件如下： apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaUser metadata:   name: my-user   labels:     strimzi.io/cluster: my-cluster spec:   authentication:     type: scram-sha-512   authorization:     type: simple     acls:       # Example ACL rules for consuming from my-topic using consumer group my-group       - resource:           type: topic           name: my-topic           patternType: literal         operation: Read         host: "*"       - resource:           type: topic           name: my-topic           patternType: literal         operation: Describe         host: "*"       - resource:           type: group           name: my-group           patternType: literal         operation: Read         host: "*"       # Example ACL rules for producing to topic my-topic       - resource:           type: topic           name: my-topic           patternType: literal         operation: Write         host: "*"       - resource:           type: topic           name: my-topic           patternType: literal         operation: Create         host: "*"       - resource:           type: topic           name: my-topic           patternType: literal         operation: Describe         host: "*"

[scholzj]

ANONYMOUS user suggests you do not have authentication enabled. But without you sharing some readable custom resource files, it is hard to guess what your configuration really is.

[dockerxu123]

I added a topic on the client side to show the success of test1. In fact, the Kafka log User:ANONYMOUS is Denied Operation = Create from host = 192.168.5.137 on resource = Topic:LITERAL:test1 for request = CreateTopics with resourceRefCount = 1 (kafka.authorizer.logger) [data-plane-kafka-request-handler-5]

…

------------------ 原始邮件 ------------------ 发件人: "strimzi/strimzi-kafka-operator" ***@***.***>; 发送时间: 2022年2月16日(星期三) 下午3:44 ***@***.***>; ***@***.******@***.***>; 主题: Re: [strimzi/strimzi-kafka-operator] The client cannot connect to Kafka cluster (Discussion #6343) ANONYMOUS user suggests you do not have authentication enabled. But without you sharing some readable custom resource files, it is hard to guess what your configuration really is. — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: ***@***.***>

[dockerxu123]

hello Jakub Scholz: kafka.yaml： apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata:   name: my-cluster spec:   kafka:     version: 3.0.0     replicas: 3     listeners:       - name: plain         port: 9092         type: internal         tls: true         authentication:           type: scram-sha-512       - name: tls         port: 9093         type: internal         tls: true         authentication:           type: scram-sha-512       - name: external         port: 9094         type: nodeport         tls: true         authentication:           type: scram-sha-512         configuration:           bootstrap:             nodePort: 32410           brokers:            - broker: 0              nodePort: 32420            - broker: 1              nodePort: 32421            - broker: 2              nodePort: 32422     authorization:       type: simple       superUsers:       - my-user       - ANONYMOUS     config:       offsets.topic.replication.factor: 3       transaction.state.log.replication.factor: 3       transaction.state.log.min.isr: 2       default.replication.factor: 3       min.insync.replicas: 2       inter.broker.protocol.version: "3.0"       ssl.endpoint.identification.algorithm: ''     storage:       type: jbod       volumes:       - id: 0         type: persistent-claim         size: 10Gi         deleteClaim: false         class: rook-rtshield-block   zookeeper:     replicas: 3     storage:       type: persistent-claim       size: 10Gi       deleteClaim: false       class: rook-rtshield-block   entityOperator:     topicOperator: {}     userOperator: {} apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaConnect metadata:   name: my-user #  annotations: #  # use-connector-resources configures this KafkaConnect #  # to use KafkaConnector resources to avoid #  # needing to call the Connect REST API directly #    strimzi.io/use-connector-resources: "true" spec:   version: 3.0.0   replicas: 1   bootstrapServers: my-cluster-kafka-bootstrap:9093   tls:     trustedCertificates:       - secretName: my-cluster-cluster-ca-cert         certificate: ca.crt   authentication:     type: scram-sha-512     username: my-user     passwordSecret:       secretName: my-user       password: password   config:     group.id: connect-cluster     offset.storage.topic: connect-cluster-offsets     config.storage.topic: connect-cluster-configs     status.storage.topic: connect-cluster-status The connection is normal, and the log is shown in the figure below

…

--- apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaConnect metadata:   name: my-user #  annotations: #  # use-connector-resources configures this KafkaConnect #  # to use KafkaConnector resources to avoid #  # needing to call the Connect REST API directly #    strimzi.io/use-connector-resources: "true" spec:   version: 3.0.0   replicas: 1   bootstrapServers: 192.168.5.137:32410   tls:     trustedCertificates:       - secretName: my-cluster-cluster-ca-cert         certificate: ca.crt   authentication:     type: scram-sha-512     username: my-user     passwordSecret:       secretName: my-user       password: password   config:     group.id: connect-cluster     offset.storage.topic: connect-cluster-offsets     config.storage.topic: connect-cluster-configs     status.storage.topic: connect-cluster-status Discovery log : SSL handshake failed ，How should this problem be solved