If a lua script in my extension has a requirement of something, will it be required when used by other Quarto projects?

[dweng0]

Description

Hi all, I have a Lua script that makes use of

local lfs = require("lfs") -- Load the Lua File System library -- TODO check documentation, does quarto support imports?

now, that's not shipped as standard and needs to be imported with lua rocks

luarocks install luafilesystem

My question is
If someone was to make use of this extension, will they need to install luarocks in order for the extension to work properly, as the script depends on it?

Supporting information

This is the extension script workflow:

It is being used so that i can check for the existence of files in a directory with a specific extension. The exact code looks like this:

-- Function to find files with specific extensions in a directory
local function find_files(directory, extensions)
    local files = {}

    for file in lfs.dir(directory) do
        if file ~= "." and file ~= ".." then
            local _, ext = file:match(".+(%..+)$")
            if ext and extensions[ext:lower()] then
                table.insert(files, file)
            end
        end
    end

    return files
end

whereby you pass in some extensions, and it returns an array of files that match the extension.

An alternative is to use ls command, but to my knowledge, that is only supported by unix like systems and i'm not sure it would work for windows (i'm on Ubuntu), something like this:

-- Commented out, will it cause windows systems to crash?
-- Function to find files with specific extensions in a directory
local function find_files(directory, extensions)
    local files = {}
    local handle = io.popen('ls "' .. directory .. '"') -- Using 'ls' command, this is only available on unix systems TODO better approach required

    if handle then
        local output = handle:read("*a")
        handle:close()

        for file in output:gmatch("[^\r\n]+") do
            local _, ext = file:match(".+(%..+)$")
            if ext and extensions[ext:lower()] then
                table.insert(files, file)
            end
        end
    end

    return files
end
``

[dweng0]

Ok after some research on ClI in windows, i see i can make it work without requiring third party tools, something like this. So maybe the question is obsolete for me, but maybe an answer might help other people who do require third party tools.

The workaround does some checks and then sets the command accordingly

    -- Use platform-specific commands to list files in the directory
    if package.config:sub(1,1) == '\\' then -- Windows
        command = 'dir /b "' .. directory .. '"'
    else -- Unix-like systems (Linux, macOS, etc.)
        command = 'ls "' .. directory .. '"'
    end

    local handle = io.popen(command)

[cderv]

If someone was to make use of this extension, will they need to install luarocks in order for the extension to work properly, as the script depends on it?

If the scripts depends on some external dependency, yes it will be necessary to have those available. Quarto will just run the Lua Script as part of the filter chain during pandoc conversion, or directly using the Lua provided by Pandoc.

So only what is available in Lua API (https://quarto.org/docs/extensions/lua-api.html) will be available without more dependencies to install.

[cscheid]

We would love to support luarocks, but that's not technically feasible.

To be more specific: in the current version of Pandoc, the Lua extension in Pandoc is statically linked, and luarocks makes extensive use of dynamic linking (including the specific lfs library in this post).

[stefanocoretta]

Hi! Just a follow up question. What if I ship the module together with the extension? (specifically, luafilesystem)

I tried the following hack, which works outside of quarto, but I get an error in run as the quarto extension.

local current_path = package.cpath
local module_folder = "./luafilesystem/?.so"
package.cpath = current_path .. ";" .. module_folder
require "lfs"

[stefanocoretta]

Thanks! I am not sure that answers my question though. In a plain Lua script it is possible to load a module from a local directory (not a module installed with luarocks), but the search path seems not to work as intended if the script is run as a filter during quarto rendering.

[cscheid]

You wrote

Thanks! I am not sure that answers my question though.

I pointed to my answer:

in the current version of Pandoc, the Lua extension in Pandoc is statically linked and luarocks makes extensive use of dynamic linking

Forgive my bluntness, but please be sure to read more carefully. That's literally the answer to your question. Filters are not plain Lua scripts. The search path doesn't find it because dynamic linking doesn't work in the statically-linked version of Lua that Pandoc ships with.

[stefanocoretta]

Bu I'm not using dynamic linking nor luarocks. Your answer says luarocks does not work with the pa doc shipped with quarto, but my use case doesn't use luarocks. The module is in the same folder as the lua filter, within the quarto extension folder, so how come the lfs.so file is not found?

[cscheid]

Pandoc's Lua cannot load that library. .so files are dynamically linked. That's what they are. That's what you're using.

[stefanocoretta]

Ah ok, now I understand. :) Thanks for the patience :) (I'm not very familiar with Lua, started learning a month ago...)