Using self signed certificates in KafkaConnect

[cstmgl]

Hi All,

My question is about how to manage the certificates in terms of java trust store when using Strimzi.
For my development cluster I'm using KafkaConnect and then an application to simulate AWS S3 (MinIO)
I have self signed certificates so I would need to put them inside the Java trust store but I could not find a way to do this.

I tried to do it in the docker image directly (with an entry point or cmd) or find other ways but it seems that it never works and my scripts never run.
Is there any idea how I could do this?
I search the documentation but most of the topics for the certificate is about the Kafka Cluster certificates not so much about the java truststore manipultion.

Any feedback is welcomed.

I'm using Strimzi version 0.14.0 and 0.18.0

[scholzj]

So, if I understand it right you want to configure the truststore in the connector? I think that depends a bit on the connector and how it accepts the certificate. You can mount the truststore using the external volumes feature: https://strimzi.io/docs/operators/latest/full/using.html#type-ExternalConfiguration-reference (not sure when we added it, but I guess it should definitely be in 0.18). But then as I said, it depends on the connector.

[scholzj]

Wouldn't be easier to allow the developer to import the SSL certificate to the default Java Key Store located in the Java Home directory? At least for my specific case, this makes the solution much easier :)

First of all, that would IMHO be very insecure and dangerous since it would mix the proper CAs shipped by the JVM with some self-signed certificates. It would also not allow to properly isolate where do you use this certificate. So I think that for a proper support of self-signed certificates or private CAs, the connector should offer a way how to configure it with custom truststore. Also, the operands run normally under user with limited privileges. So they should not be able to modify any files in the JVM installation such as the default CA store.

I can understand that if you have to use a connector that does not offer anything better you might have no choice. But I think in that case actually using a custom container image where you modify this might be the best solution.

[inpt333]

First of all, that would IMHO be very insecure and dangerous since it would mix the proper CAs shipped by the JVM with some self-signed certificates. It would also not allow to properly isolate where do you use this certificate. So I think that for a proper support of self-signed certificates or private CAs, the connector should offer a way how to configure it with custom truststore. Also, the operands run normally under user with limited privileges. So they should not be able to modify any files in the JVM installation such as the default CA store.

Yes, that's a good point. I was thinking more on supporting self-signed certificates for a dev environment, rather than productive environment.

By the way, sorry, I was misleading you by using the concept KeyStore, instead of TrustStore.

[sudoduan]

@cstmgl Has your issue been resolved? We have a similar setup where we use minio, with a self signed cert, as the sink. We use io.confluent.connect.s3.S3SinkConnector for the kafka connector. Would you mind share your experience? Thanks!

[cstmgl]

I'm not sure I would recommend our solution.
We just went with importing the certificates to the JVM keystore.

There is a way to load certificates also but I have not tried it myself.
We basically change the script to start the connector to load all certificates of a certain mount point to the JVM and then initialize the actual connect cluster

@cstmgl Has your issue been resolved? We have a similar setup where we use minio, with a self signed cert, as the sink. We use io.confluent.connect.s3.S3SinkConnector for the kafka connector. Would you mind share your experience? Thanks!

[sudoduan]

@cstmgl Thank you for sharing. We will try it out.

[rogeriomm]

My solution utilizes Kafka Strimzi 0.39. It is independent of the connector, but it is less secure:

https://github.com/rogeriomm/labtools-k8s/blob/a605556d1e1e029b005dbc7b97a81ba6a0d10535/install/kafka/connectors/Dockerfile#L129
https://github.com/rogeriomm/labtools-k8s/blob/a605556d1e1e029b005dbc7b97a81ba6a0d10535/install/kafka/connectors/update-ca-trust.sh
https://github.com/rogeriomm/labtools-k8s/blob/a605556d1e1e029b005dbc7b97a81ba6a0d10535/k8s/cluster2/base/kafka/main/connect.yaml#L61