Kafka VM Parameters

[AsimDandapat]

Hi Team & specially @scholzj
For Apache kafka on vmware we normally update VM configuration for higher performance.
So please let me know how this has been taken care in Strimzi kafka pod.

set_security_limits
/etc/security/limits.conf
hard nofile 100000
soft nofile 100000

vm_swappiness: Sysctl option vm.swappiness set to be changed to 1
net.core.somaxconn = 400

[scholzj]

Long time since I looked into this. But I think this is controlled by the worker node.

[AsimDandapat]

Hello thank you for the quick reply.
And, just for my understanding can you please guide me how important this parameters so I can check with cloud team.

[scholzj]

I'm not sure I'm the right person for this. The open file limits are obviously important. But they will not improve performance or something. You just need to have enough of them. How many, that depends on how many partition replicas you have, how big your segments are, how many of them you have etc. These all affect the number of open files.

I also think that usually, Kubernetes recommends disabling swap. I think that is a good practice for Kafka it self as well.

I have no idea about net.core.somaxconn -> googling suggests it is about a rate with which new connections are created? With Kafka I would normally expect that the connection should be more stable and long term. So it might not be important. But I don't really know about this one TBH.

[AsimDandapat]

Thank you @scholzj :)