We still need the Lua transform

[hunsbea]

This is not an issue per se, but https://vector.dev/docs/reference/configuration/transforms/lua/ has a warning banner encouraging me to tell you if any Lua use cases are not covered by the VRL remap transform.

We are doing something like this, to run arbitrary system commands only once per log file to enrich log metadata with the owner of the log file, and the process writing to the log file. We plan to extend this further to get more specific metadata related to our environment with additional commands.

transforms:
  enrich_logs:
    type: lua
    inputs:
      - app_logs
    version: "2"
    # This will only run the expensive system commands the first time we see a log file, then we'll use the cached results
    source: |
      function get_file_owner(filepath)
        local handle = io.popen("stat -c '%U' '" .. filepath .. "' 2>/dev/null")
        local user = handle and handle:read("*a"):gsub("%s+", "") or "unknown"
        if handle then handle:close() end
        return user
      end

      function get_process_name(filepath)
        local handle = io.popen("fuser '" .. filepath .. "' 2>/dev/null | xargs -r ps -o comm= -p | grep -v vector | head -1")
        local result = handle and handle:read("*a"):gsub("%s+", "") or ""
        if handle then handle:close() end
        return result
      end

      function init(emit)
        file_metadata_cache = {}
      end

      function process(event, emit)
        local filepath = event.log.file
        if filepath and not file_metadata_cache[filepath] then
          local user = get_file_owner(filepath)
          local process_name = get_process_name(filepath)
          file_metadata_cache[filepath] = {
            process_name = process_name,
            owner_user = user
          }
        end

        if filepath and file_metadata_cache[filepath] then
          local metadata = file_metadata_cache[filepath]
          event.log.process.name = metadata.process_name
          event.log.file.owner_user = metadata.owner_user
        end

        emit(event)
      end
    hooks:
      init: "init"
      process: "process"

If I could run arbitrary system commands and store a global cache state using VRL, I would happily switch to that. I don't need any immediate assistance here, but I would just say please don't deprecate the Lua transform without allowing this kind of thing in VRL.