How can I support execution of the Rust programming language in Quarto?

[galenseilis]

Description

I am hoping to blog about my journey of learning Rust. I have tried to learn some of the basics of Lua to produce a filter, but my lack experience with Lua and Pandoc in the back end of Quarto is proving a challenge.

Here is my Lua filter. It has some logging stuff to help me figure things out, but I'll probably remove them later.

local io = require("io")
local os = require("os")
local tempfile = require("os").tmpname
local log_file

-- Function to initialize the log file
local function init_log()
  log_file = io.open("rust_executor_debug.log", "w")
end

-- Function to log messages to file and stderr
local function log(...)
  local args = {...}
  for i = 1, #args do
    args[i] = tostring(args[i])
  end
  local message = table.concat(args, " ")
  if log_file then
    log_file:write(message .. "\n")
    log_file:flush()
  end
  io.stderr:write(message .. "\n")
  io.stderr:flush()
end

-- Helper function to execute Rust code and return the output
local function execute_rust_code(code)
  local temp_file = tempfile() .. ".rs"
  log("Temporary Rust file:", temp_file)
  local source_file, err = io.open(temp_file, "w")
  if not source_file then
    log("Failed to create source file:", err)
    error("Failed to create source file: " .. err)
  end

  source_file:write(code)
  source_file:close()

  local temp_bin = tempfile()
  log("Temporary binary file:", temp_bin)

  local compile_command = "rustc " .. temp_file .. " -o " .. temp_bin .. " 2>&1"
  log("Compile Command:", compile_command)
  local compile_pipe = io.popen(compile_command)
  local compile_output = compile_pipe:read("*a")
  local compile_result = compile_pipe:close()

  if compile_result ~= true then
    os.remove(temp_file)
    log("Rust compilation failed. Output:", compile_output)
    error("Rust compilation failed. Output: " .. compile_output)
  end

  local exec_command = temp_bin .. " 2>&1"
  log("Exec Command:", exec_command)
  local exec_pipe = io.popen(exec_command)
  local output = exec_pipe:read("*a")
  exec_pipe:close()

  local ok, rm_err = pcall(function()
    os.remove(temp_file)
    os.remove(temp_bin)
  end)
  if not ok then
    log("Failed to clean up temporary files:", rm_err)
    error("Failed to clean up temporary files: " .. rm_err)
  end

  log("Output:", output)
  return output
end

local echo_global = true

function Meta(meta)
  if meta.echo ~= nil then
    echo_global = pandoc.utils.stringify(meta.echo) == "true"
  end
end

-- Lua filter function
function CodeBlock(elem)
  if not log_file then
    init_log()
  end

  local is_rust_code = elem.attr.classes:includes("{rust}")
  if is_rust_code then
    log("Processing Rust code block")
    local output = execute_rust_code(elem.text)
    output = output:gsub("%s+$", "")
    local blocks = {}

    if echo_global then
      -- Render Rust code as a formatted block
      table.insert(blocks, pandoc.CodeBlock(elem.text, {class="rust"}))
    end

    -- Always return the output
    table.insert(blocks, pandoc.Para(pandoc.Str(output)))

    return blocks
  else
    log("Skipping non-Rust code block")
  end
end

-- Ensure log file is closed properly at the end
function Pandoc(doc)
  if log_file then
    log_file:close()
  end
  return doc
end

Here is my Quarto blog post. I recommend looking at the plain markdown as Github's preview is suggesting that my Rust code blocks are breaking up the preview...

---
title: "Executable Rust Code in Quarto"
author: "Galen Seilis"
date: "2024-07-26"
categories: [Quarto, Rust, Lua, Pandoc]
echo: false
---

::: {.callout-warning}
## WIP

This post is a work in progress. I am experimenting with rendering Rust code. It still needs some work.
:::


Here is some Rust code that will be executed and rendered.

```{rust}
fn main() {
        println!("Galen Seilis is learning Rust!");
        println!("Time to get Rusty!");
}

Now let us try some Rust code that will not be executed.

fn main() {
    println!("Meow");
}

Now let us run a longer example from Rust by Example.

fn main() {
    // Integer addition
    println!("1 + 2 = {}", 1u32 + 2);

    // Integer subtraction
    println!("1 - 2 = {}", 1i32 - 2);
    // TODO ^ Try changing `1i32` to `1u32` to see why the type is important

    // Scientific notation
    println!("1e4 is {}, -2.5e-3 is {}", 1e4, -2.5e-3);

    // Short-circuiting boolean logic
    println!("true AND false is {}", true && false);
    println!("true OR false is {}", true || false);
    println!("NOT true is {}", !true);

    // Bitwise operations
    println!("0011 AND 0101 is {:04b}", 0b0011u32 & 0b0101);
    println!("0011 OR 0101 is {:04b}", 0b0011u32 | 0b0101);
    println!("0011 XOR 0101 is {:04b}\n\n\n", 0b0011u32 ^ 0b0101);
    println!("1 << 5 is {}", 1u32 << 5);
    println!("0x80 >> 2 is 0x{:x}", 0x80u32 >> 2);

    // Use underscores to improve readability!
    println!("One million is written as {}", 1_000_000u32);
}

There are a couple issues I am experiencing. The first is that even when I have set `echo: false` in the preamble I am finding that the Rust code in code section with a `{rust}` is still rendering. Similar to rendering the output of other languages, I would like to be able to turn this on and off both for the whole QMD page as well as locally within the code block. Even just having one or the other would be better than not having some control over rendering, though.

![image](https://github.com/user-attachments/assets/a454a53e-57bc-4a6c-89cb-41a5fe4110e3)

I have not looked into the second issue yet, but I'll mention it anyway. I expect I just need to include a newline character in my Lua filter, but I am finding that the output from the executed Rust code is not including any newlines:

![image](https://github.com/user-attachments/assets/394a68c2-595d-4da0-85d0-0ac7e9f4e7fe)

I would appreciate some guidance on next steps for debugging or implementing the desired behaviour.

[galenseilis]

The current state is located here: https://galenseilis.github.io/blog/posts/rust-execution-for-quarto/

[mcanouil]

Did you try the Jupyter kernel for rust: https://github.com/evcxr/evcxr

[galenseilis]

I have not! That looks promising.

[galenseilis]

I tried this option. Turned out to be easy and pretty much what I wanted.

[cderv]

Other discussion on this topic

• How do I get Rust execution in my Quarto blog? #10399

Additionally to a Rust jupyter engine, it exists R package allowing to call Rust and offers a knitr engine that could be used in Quarto. package is https://extendr.github.io/rextendr/index.html

See more : #10399 (comment)