Getting "TLS termination for host 'mycluster-broker-1.local' requires specifying a TLS secret or configuring a global wildcard TLS secret"

[itaydj]

Hi,
I am trying to configure my Strimzi external listener to ingress with tls enabled and our own certificate using cert manager.
we need ssl passthrough as well so that the brokers are also tls enabled.
seems like everything is configured properly except that im getting the following errors when I run
command

kubectl get events -n mynamespace

error:

"TLS termination for host 'strimzi-bootstrap.strimzi-sanity-tests.develop.mydomain.local' requires specifying a TLS secret or configuring a global wildcard TLS secret"

I believe the secret is already configured in my Kafka resource and its pointing to a valid certificate:

apiVersion: v1
items:
- apiVersion: kafka.strimzi.io/v1beta2
  kind: Kafka
  metadata:
    annotations:
      meta.helm.sh/release-name: itay-kafka
      meta.helm.sh/release-namespace: strimzi-sanity-tests
    generation: 2
    labels:
      app.kubernetes.io/managed-by: Helm
    name: strimzi
    namespace: strimzi-sanity-tests
  spec:
    kafka:
      authorization:
        superUsers:
        - admin-kafka-user
        type: simple
      config: {}
      listeners:
      - authentication:
          type: scram-sha-512
        configuration:
          bootstrap:
            host: strimzi-bootstrap.strimzi-sanity-tests.develop.mydomain.local
          brokerCertChainAndKey:
            certificate: tls.crt
            key: tls.key
            secretName: strimzi-strimzi-tls
          brokers:
          - broker: 0
            host: strimzi-broker0.strimzi-sanity-tests.develop.mydomain.local
          - broker: 1
            host: strimzi-broker1.strimzi-sanity-tests.develop.mydomain.local
          - broker: 2
            host: strimzi-broker2.strimzi-sanity-tests.develop.mydomain.local
          class: nginx
        name: external
        port: 9094
        tls: true
        type: ingress
      - authentication:
          type: scram-sha-512
        name: internal
        port: 9093
        tls: true
        type: internal

also,
i get the following when running :

$ openssl s_client -connect strimzi-bootstrap.strimzi-sanity-tests.develop.mydomain.local:443 -servername strimzi-bootstrap.strimzi-sanity-tests.develop.mydomain.local

CONNECTED(00000005)
4001C2E201000000:error:0A000458:SSL routines:ssl3_read_bytes:tlsv1 unrecognized name:ssl/record/rec_layer_s3.c:1605:SSL alert number 112
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 368 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

the ingress role of all brokers is indeed missing the "secretName" key under tls by the way even though it is specified in the kafka resource:

  apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      field.cattle.io/publicEndpoints: '[{"addresses":["172.25.10.247"],"port":443,"protocol":"HTTPS","serviceName":"strimzi-sanity-tests:strimzi-kafka-0","ingressName":"strimzi-sanity-tests:strimzi-kafka-0","hostname":"strimzi-broker0.strimzi-sanity-tests.develop.mydomain.local","path":"/","allNodes":true}]'
      ingress.kubernetes.io/ssl-passthrough: "true"
      nginx.ingress.kubernetes.io/backend-protocol: HTTPS
      nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    creationTimestamp: "2023-05-29T13:27:07Z"
    generation: 1
    labels:
      app.kubernetes.io/instance: strimzi
      app.kubernetes.io/managed-by: strimzi-cluster-operator
      app.kubernetes.io/name: kafka
      app.kubernetes.io/part-of: strimzi-strimzi
      strimzi.io/cluster: strimzi
      strimzi.io/kind: Kafka
      strimzi.io/name: strimzi-kafka
    name: strimzi-kafka-0
    namespace: strimzi-sanity-tests
    ownerReferences:
    - apiVersion: kafka.strimzi.io/v1beta2
      blockOwnerDeletion: false
      controller: false
      kind: Kafka
      name: strimzi
      uid: 6275aaa7-e13d-42eb-b511-7b32b467d682
    resourceVersion: "4151500835"
    uid: 25505e35-0ec3-415d-84d4-504093c0dcdf
  spec:
    ingressClassName: nginx
    rules:
    - host: strimzi-broker0.strimzi-sanity-tests.develop.mydomain.local
      http:
        paths:
        - backend:
            service:
              name: strimzi-kafka-0
              port:
                number: 9094
          path: /
          pathType: Prefix
    tls:
    - hosts:
      - strimzi-broker0.strimzi-sanity-tests.develop.mydomain.local

Im trying to understand what am I missing here...
any idea are more than welcome.
thanks

[scholzj]

I guess that suggests your TLS passthrough is not really working in the Ingress? As the event suggests it is trying to do TLS termination. The secret is indeed missing in the Ingress resources -> because with TLS passthrough it is not needed since the TLS traffic has to go through ingress into the broker without being decoded in any way.

[itaydj]

oh...I see,
Ill check and update this discussion soon

[itaydj]

well, we are using nginx plus ingress controller and not the open source one.
in order to support ssl passthrough, I need to configure an annotation on the ingress resource.
Is the a way to add a custom annotation on each ingress rule?

the annotation I need to add is the following:

annotations:
    nginx.org/ssl-services: "mystrimzi-broker-0"

for each broker and for the bootstrap ingress rule

[scholzj]

You should be able to set the annotations in the listener configuration -> see for example here: https://strimzi.io/docs/operators/latest/full/configuring.html#property-listener-config-annotations-reference

[itaydj]

Thank you very much, I'll try that and update for future reference.

…

On Mon, May 29, 2023, 17:16 Jakub Scholz ***@***.***> wrote: You should be able to set the annotations in the listener configuration -> see for example here: https://strimzi.io/docs/operators/latest/full/configuring.html#property-listener-config-annotations-reference — Reply to this email directly, view it on GitHub <#8580 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD2FI2KXJZFWSPD5MTLXJ3TXISVS3ANCNFSM6AAAAAAYS2N7RI> . You are receiving this because you authored the thread.Message ID: <strimzi/strimzi-kafka-operator/repo-discussions/8580/comments/6029749@ github.com>