Cannot use lfs (lua file system) in LUA filter plugin.

[maitry-nutanix]

PROBLEM

I need to find the size of a log file in lua script. I can do that by opening and reading the file (using the code below) but that doesn't seem like a good solution as this script runs for every log line read by fluenbit.

        local file_size = file:seek('end')
        file:close()

CODE

I cannot use lfs in LUA script in LUA filter plugin. I used:

local lfs = require "lfs"
local size = lfs.attributes (path, "size")

ERROR

But getting an error as follows:

[2023/07/25 13:44:43] [error] [filter:lua:lua.1] error code 2: /fluent-bit/etc/..2023_07_25_13_44_22.370465307/filters.lua:6: module 'lfs' not found:
	no field package.preload['lfs']
	no file './lfs.lua'
	no file '/usr/local/share/luajit-2.1.0-beta3/lfs.lua'
	no file '/usr/local/share/lua/5.1/lfs.lua'
	no file '/usr/local/share/lua/5.1/lfs/init.lua'
	no file './lfs.so'
	no file '/usr/local/lib/lua/5.1/lfs.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'

HELP NEEDED

1. Is there any other way to use lfs or it is not supported yet.
2. Open for any other solutions instead of using lfs. Please share if you have any ideas.

[patrick-stephens]

LUA modules may need system dependencies so these need to be installed. It looks like you need to install the dependencies required.

[arunim2405]

@jadiym can you please share the dockerfile that you ended up with? We need to implement something similar where we need to use lua-zlib.
@patrick-stephens I also tried using a compile luac.out file as a script but that unfortunately did not work. Do you see it as a valid feature request ? (support for compiled lua files)

[patrick-stephens]

I don't think we support compiled lua files but depends on what LuaJIT does.

[tchrono]

@arunim2405 fluent-bit simply embeds LuaJIT engine. If LuaJIT supports loading precompiled modules (it should support), then it should work under fluent-bit.

Here's an article that might help you: https://blog.openresty.com/en/luajit-bytecode/

[tchrono]

BTW I'm not sure LuaJIT bytecode is portable across versions. If you do this, make sure to use the exact same version of LuaJIT used by fluent-bit.

[patrick-stephens]

It seems a brittle way to do it to be honest