mTLS failure - Produced client Certificate message is empty

[supermom5280]

Using Strimzi 36.1 Kafka 3.5 Java 17 (on client). When the CertificateRequest message is received is only has the Strimzi CA in the list. Where do I add the CA for the client so that when the request comes from the Server it has both the Strimzi CA and my CA?

[scholzj]

Without any logs and more detailed explanation, it is not really clear about what part of the TLS handshake are you talking. If you want to use your own (custom) CA, you have to provide it instead of the Strimzi CA. There is a whole section in the docs dedicated to it. If you want, you can also provide only your own server certificate for a specific listener.

[supermom5280]

unfortunately the system is air gapped so providing logs is impossible but I will take a stab at providing enough details. The Listener has it own cert and authorizing without issue. When I create a user using authentication type = tls and allow Strimzi to create the cert for the user I am able to download the the certs and add them to the client p12 and truststore works great ... client is able to communicate and put messages on the top. When I attempt to use certs signed outside of Strimzi the handshake fails ... Strimzi/kafka request send a Certificate Request
"Certificate Request" :{
"certificate_request_context" : "",
"extensions":[
"signature algoithms": [

"certificate_authorities": {
"certificate authorities" :[
CN=clients-ca v0, O=io.strimzi]
}
i have tried added the outside client CA to the -clients-ca-cert / ca.crt also to the user secret ca.crt but get the same result. The client can not find a cert that is signed by clients-ca so it return and empty cert and strimzi replies with a "bad certificate"

[supermom5280]

I have changed the user from authentication type tls to tls-external

[scholzj]

Ok, so we are talking about the Clients CA for mTLS. That helps, no logs needed right now.

The tls-external authentication in KafkaUser allows you to configure the users for ACLs or Quotas, but issue the mTLS certificate yourself out-of-band. But it will not change the CAs trusted by the broker. That can be done only using the custom CA.

Do you have a reason why you want to use both the Strimzi CA and your custom CA in parallel? Strimzi does not really support that easily right now. If the custom CA is all you need, you can follow https://strimzi.io/docs/operators/latest/full/deploying.html#proc-replacing-your-own-private-keys-str to replace the Strimzi Clients CA with the custom CA.

If you want to use both in parallel, you can add your CA public key to the Secret. But you might need to do a manual rolling update of the Kafka brokers to get it loaded. I'm also not sure what will happen to it when the Strimzi's own Clients CA is renewed.

[supermom5280]

Thank you ... I had spent a ton of time trying to follow how the get the CA replaced with no success ... so I was trying minimize by simply adding the client CA while allowing Stimzi to continue to manage all the certs for all the other operators (zookeeper, topic operator etc) within the cluster. If I may ask 2 clarifying questions, when you say "you can add your CA public key to the Secret" which secret is that? Also, when using outside signed certs for a user, there is not a need to create a secret named the same as the user created using user-operator?

[scholzj]

while allowing Stimzi to continue to manage all the certs for all the other operators (zookeeper, topic operator etc) within the cluster.

The internal Strimzi communication does not use the Clients CA, only the Cluster CA. So if you want to use your own Clients CA, it will have no impact on it. The Clients CA is used only for your clients.

when you say "you can add your CA public key to the Secret" which secret is that?

This would be the <cluster-name>-clients-ca-cert Secret.

Also, when using outside signed certs for a user, there is not a need to create a secret named the same as the user created using user-operator?

No. The trust is done through the CA and not through the client certificate. So you do not need to create the Secret. But keep in mind that to use the User Operator this way, the certificate has to follow the subject rules (i.e. have the subject CN=<username>).

[supermom5280]

Thank you for clarifying some points. I will modify the clients-ca-cert and ensure the the CN= is exactly as it appeared in the kafka-user.yml