Order of Operations in quarto render

[andrewpbray]

Description

Good morning!

I've been working on filters and pre-render scripts for single documents and projects, and it'd be very helpful to have a finer-grained sense of where these pop up into the flowchart of what happens during quarto render on the road from .qmd to final output file.

For both a single document render an a project render, what is the order of the different operations? My inference based on the docs and my own experimentation is something like:

Single Document Render

1. Collects any meta data from other files and merges with document metadata
2. Parses qmd into an AST
3. Pulls out code cells and send them to jupyter/knitr for evaluation and collection of outputs
4. Runs Quarto filters (callout blocks, layouts, etc) and decorates AST with appropriate attributes
5. Writes AST into .md file
6. Sends .md file to pandoc to write to output format

My questions:

1. I'm sure this is wrong/incomplete. Could someone flesh it out correctly or point me to where I can look to try to figure it out?
2. Where in this process does a lua filter extension get run? (e.g. between 5 and 6) How does this change if you change the order so that it runs after - quarto.
3. Where in this process do pre- and post-render scripts get run?

I have similar questions for a project.

Website Render

1. Build list of all qmd files to be rendered subject to render target globs
2. Build any listings JSONS
3. Run pre-render scripts.
4. Etc.

[cscheid]

I don't have a great answer for you, but I want to acknowledge that we really do need to write an internals guide to Quarto.

I think we won't escape writing one in 1.5 (since there are bigger structural changes coming), but I don't think we'll be able to do a good enough job before then.

[andrewpbray]

Got it 👍 .

One reason why this has come up for me: I've been working on a course website that has some extra functionality built in via R scripts that read from a _course-settings.yml file that then modify the _quarto.yml (essentially pre-pre-render scripts). If you a) gave users more control over where their filters were run and b) shipped a yaml parser with the lua distribution (don't know how trivial this is) then it seems like we're getting close to being able to write Project Type Extensions of the form

project:
  type: andrews-website

that would wrap this sort of scripting functionality into a much more portable/sharable form.

[cscheid]

b) shipped a yaml parser with the lua distribution (don't know how trivial this is)

I'm sure someone has written one, but FWIW, the reason we don't do it is that we don't actually read YAML in Lua ourselves. With that said, if you need this in a pre-render script, then you could write your script in TypeScript, for which we do ship a YAML parser.

[andrewpbray]

Ahh, ok, good to know. I'll start poking around that option.

Thanks!

[mcanouil]

A somewhat related discussion:

• On the order of things with regards to pandoc's AST #541