Using Kafka connector with PEM certificates not working

[Modernaddis]

Hello, I am trying to connect strimzi kafka cluster with elastic search using Kafka connect and Kafka connector resource. I am able to get a successful connection using JKS type for keystore and truststore. However I have to convert the TLS certificates to JKS manually for that to work.
I would like to know if it is possible to use the tls certificates(tls.key,tls.crt,ca.crt) in kafka connector without converting to JKS?
Using io.confluent.connect.elasticsearch.ElasticsearchSinkConnector as the class for the connector.

[scholzj]

I assume you are talking about the connection to Elastic Search? That depends on the connector you are using (and probably on the library it is using to talk with Elastic Search).

[Modernaddis]

@scholzj Thanks for your reply. It is using io.confluent.connect.elasticsearch.ElasticsearchSinkConnector as the class for the connector. Clients are able to connect to external listeners using just the TLS certificates without any conversion to JKS. My question is can't Kafka connector CRD do the same to connect to Elastic Search?
Here are some docs from Apache Kafka that show the feature being implemented but can't see any reference about it on Strimzi's side..

[scholzj]

I don't think you replied to the first question I asked. About what side of the connector are you talking about. The Kafka part is handled by Strimzi, so I would assume you talk about the ElasticSearch part of the connector. In which case it does not matter about what Kafka supports. It is only about what ElasticSearch and the connector support.

[Modernaddis]

I understand what you mean by "it's about what Elastic search supports". But since we're configuring a Kafka connector resource, I thought there was some control on that end on what can be specified and what can't. I'll look into the ES side of things..

[scholzj]

Strimzi can handle only the Kafka side of things. What is supported on the other side has basically nothing to do with Kafka. That is about some client for the other system. And this will be ultimately different for every connector.

[Modernaddis]

Understood. Thank you @scholzj

[scholzj]

BTW: I created a new a Cofngi Provider which might be able to help with this: https://github.com/scholzj/kafka-x509-config-provider ... it can create an PKCS12 store which normally works with Java on the fly from the PEM files. So if you still need this, you can give it a try.

[Modernaddis]

Last question, I have a Kafka connector that fails sometimes depending on the type of data that is in the topic. What is the best way to make it fault tolerant so that it can continue to process new messages even when a task fails in the connector?

[Modernaddis]

@scholzj

[scholzj]

I don't know. That will for sure depend connector by connector, what type of failure it is and so on.