kafka connect jvm questions

[cstmgl]

Hi all,
I hope I'm not in the wrong topic to discuss this but I would like to know how to handle better the java JVM memory and some issues I'm having where I get processes not able to start because of OOM issues on my Kafka Connectors.

I was reading for example this: https://stackoverflow.com/questions/50621962/how-to-set-kafka-connect-connector-and-tasks-jvm-heap-size and some other articles I found about tweaking the flush sizes and other things like that when I noticed something I didn't fully understand on my java processes so I decided to come and ask.

Basically I want to understand if the Strimzi is supposed to handle the java -Xms and -Xmx for me or if I should configure it myself.
I'm asking because I want to make sure I don't have a clash between my cpu and memory limits and the values of the JVM.
Let's say for example I set my resources memory to 4GB and my cpu to 2 I want to make sure no issues come up from the memory restrictions on the pod instead of the operating system.

I noticed I have 2 connectors in different projects and it seems Strimzi is managing this because I see different values in it but I just want to be sure.
For example I see:

sh-4.2$ ps -ef | grep java
1001480+      85      27  5 00:20 ?        00:27:11 java -Xms8589934592 -Xmx8589934592

sh-4.2$ ps -ef | grep java
1005330+     106      28  3 Feb15 ?        01:47:28 java -Xms17179869184 -Xmx1717986918

And my limits are 8Gi and 16Gi respectively which kind of seem to match (I assume the values are in byte) only a bit higher then the limits defined.

Anyway if Strimzi is managing this is there any good guide I can find to try and understand from where my out of memories might be coming from?

[scholzj]

Strimzi tries to set some values. But I do not think you can say that it will be managing this for you. It really just sets some defaults based on the resource request / limits. Given that each connector will be different and might require different settings, you should feel free to set the -Xmx / -Xms values your self instead of relying on Strimzi. You can do it in the KafkaConnect custom resource.