Stop monitoring inactive file

[CNicer]

My scenario involves multiple log files with new content at the same time. These files stop being modified at different times and are guaranteed not to be modified again. However, these unmodifiable files are neither renamed nor deleted. I've noticed that fluent-bit is constantly holding onto the file directories of these files, preventing memory usage from decreasing.
I'm wondering if there's a way to stop monitoring inactive files after, say, 1 minute or 1 hour.

[patrick-stephens]

Please follow the issue template and provide the config/version/OS/etc.

I'm guessing you're talking about tail input but maybe not, and which OS or filesystem you are on would impact things too.

[patrick-stephens]

I think the ignore_older option for tail does not release them but just ignores them if they exist @lecaros ?

https://docs.fluentbit.io/manual/data-pipeline/inputs/tail

[lecaros]

it will at least keep them in the ignored files list. This is expected behavior so if any file is later modified it will know how to handle it. You may want to try with ignore_active_older_files (plus ignore_older, not instead of)

[patrick-stephens]

Useful to know as it's not in the docs but should show via -i tail --help I think

[CNicer]

Sorry. I'm using Tail input.
OS: Linux
Indeed, I didn't find any documentation for "ignore_active_older_files." But I found it using -i tail --help . I'll try it now. Thanks.

[CNicer]

it will at least keep them in the ignored files list. This is expected behavior so if any file is later modified it will know how to handle it. You may want to try with ignore_active_older_files (plus ignore_older, not instead of)

Very useful, thank u

[patrick-stephens]

Be good to update the docs, feel free to send a PR to the fluent-bit-docs repo

[eschabell]

@CNicer please do send over a docs PR https://github.com/fluent/fluent-bit-docs/pulls or raise an issue https://github.com/fluent/fluent-bit-docs/issues/new with how to drop active FDs properly in tail and I'll pick it up.

[CNicer]

@CNicer please do send over a docs PR https://github.com/fluent/fluent-bit-docs/pulls or raise an issue https://github.com/fluent/fluent-bit-docs/issues/new with how to drop active FDs properly in tail and I'll pick it up.

I have created a new PR, please help me take a look: fluent/fluent-bit-docs#2126