Fluentd/Fluentbit bufferStorageVolume OOMKilled

[HamzaZo]

Hi,

I'm trying to deploy/use the logging-operator in our baremetal environment with no persistentVolume only emptyDir and hostPath are allowed, and I have a few questions about it :

• Does fluentd/fluentbit need a bufferStorageVolume in my case? I tried the option disablePvc: true and without specifying any bufferStorage I got some errors in fluentbit like no upstream connections available and connection refused on port so no logs are shipped to fluentd.

• Does emptyDir a good choice? if so where fluentd store its chunks ?

I noticed when I used hostPath /var/log/fluentbit for fluentbit it consumes a lot of memory which leads to so many OOMKilled from the kernel, when I check the content of the hostPath I found so many files being created with no stop under /var/log/fluentbit/tail.0

Thanks

[pepov]

If you need a durable solution, than you will need to configure your fluentd deployment with hostPath bufferStorageVolume. I you use emptyDir you can lose messages - even if you configure file buffers - when the pod terminates.

If you fix your fluentd deployment, the fluentbit issue might get resolved as well (since it will be able to flush the buffers)