complex thesis with quarto

[fredguth]

I am trying to migrate my thesis to quarto, but I still don't get the best way to do that. I have a custom .cls, .sty and .tex files. Obviously I want to transform most .tex files (chapters) into .qmd files. Still, I don't know what to do with my main.tex which loads everything, including external packages.

As far as I understood, quarto-cli creates a temporary index.tex file and deletes it after using. How do I customize this file to add the packages that I need?

Do I need to create an extension for this kind of customization? Where do I find more in-depth documentation?

[rgaiacs]

Hi @fredguth Thanks for your enthusiastic adoption of Quarto.

Quarto processes the .qmd file and converts it to .md that Pandoc converts to the final format (.pdf built in LaTeX in your case). Quarto has a couple of filter not available at vanilla Pandoc. Pandoc manual mentions the option --include-before-body that is used to add custom LaTeX.

-B FILE, --include-before-body=FILE|URL

Include contents of FILE, verbatim, at the beginning of the document body (e.g. after the tag in HTML, or the \begin{document} command in LaTeX). This can be used to include navigation bars or banners in HTML documents. This option can be used repeatedly to include multiple files. They will be included in the order specified. Implies --standalone.

This option is available in Quarto, see https://quarto.org/docs/reference/formats/pdf.html#includes.

[cderv]

Thanks for you answer @rgaiacs. Let me give some additional context and details.

This will be a long answer to cover everything I think off, and I hope it will help you get a broad picture of what you can do.

Quarto is based on Pandoc (https://pandoc.org) and use its system with useful addition. The conversion from .qmd to .pdf, will pass through a .md file (with computational code evaluated) converted to .tex by Pandoc going through an abstract representation (AST) and using a default template. Then LaTeX will convert to PDF.

You can interact with the process is several way to tweak the default.

First, as I mentioned, a default TeX template is used. You can find it [here for Pandoc]https://github.com/jgm/pandoc/blob/master/data/templates/default.latex) and we get a copy of it in Quarto. This template follows Pandoc's templating system (https://pandoc.org/MANUAL.html#template-syntax) which allows some variables. This mean of lot of things can be tweak by only setting some value in the YAML, like setting title, changing documentclass, changing fonts, changing geometry, or using natbib or biblatex. See reference page of Quarto for all the available option you can tweak (for Pandoc template and more feature): https://quarto.org/docs/reference/formats/pdf.html. The PDF format guide will also contains some useful information. Note that the template already contains some package to load.

More specifically, you can also add some content in header, before body and after body using special includes value in Quarto https://quarto.org/docs/output-formats/pdf-basics.html#latex-includes
that is useful to add some specific content to the default template without changing it.

What to do if your template is more complex that the default one and setting YAML fields or setting includes is not enough ?
With Pandoc only, you need to provide your own full template, but in Quarto we improve that by allowing modifying only some partial template. This is documented in the Journals Article section (i'll explain more on this later):

• template: allows to pass a full new template to use : https://quarto.org/docs/journals/templates.html#templates
  This can be useful, but you would need to maintain the whole template. Some Pandoc and Quarto features may not work if you omit some part (like syntax highlighting for example).
• template-partials: allows to completely change part of the default template (https://quarto.org/docs/journals/templates.html#template-partials). This can be useful to replace part of the default template, or add more partials. Partials are different than includes because they are Pandoc's template file (they can use new variables, or existing one).

So for complex document, you would tweak this. For example doing that in your YAML header

format:
  pdf:
    template-partials:
      - title.tex

This would replace the title part of the default template, by your own template.

Another way to tweak content is to do some processing before the template to transform the parsed Markdown content to some specific LaTeX code to be inserted (usually modyfing the body part or using the metadata). This can be done with Lua using a Filter. See more on this here: https://quarto.org/docs/extensions/filters.html

All this can be done for a single project.

The step after that would be to create a new format for Quarto, instead of tweaking the default pdf one. This is possible using Quarto extensions mechanism, to create a new format: https://quarto.org/docs/extensions/formats.html
And this is what we use to offer Article for Journal using Quarto: https://quarto.org/docs/journals/
You can look at example of what creating a new format for LaTeX looks like in the sources: https://github.com/quarto-journals/

It would use the same mechanism as described above (providing a new template, or rather using new partials, and possibly some filters) but this could be used in several project by installing the extension and then using format: custom-pdf for example.

Hope it helps you find the right way to use Quarto for your thesis. (We don't have a thesis format yet I think...)

[fredguth]

@cderv,
since you posted this very complete answer, I have been using Quarto with different projects and getting used to it and I can say that I already love it.

I thought I would need to explore the nuts and bolts of Quarto and pandoc, develop lua-filters etc... but most customizations I needed can actually be easily done with Quarto/Pandoc variables (as you predicted). Using variables and partials I believe I can finish tweaking a book project to my thesis format.

Still, I was looking on the repo you referenced https://github.com/quarto-journals/. Is there anything different for creating an extension for a book?

[fredguth]

@dragonstyle and @cderv, I am in the process of building abnt-thesis based on the proof-of-concept in https://github.com/dragonstyle/abnt and I am facing some difficulties and questions:

1. user input: the only way I know to get data from the user and inserting in the document is by the use of metadata. This is not ideal. Instead of inserting an epigraph (for example) in the yaml, it would be better if the user just edited my frontmatter/epigraph.qmdfile and I insert that in the latex template (similar to what happens with chapters). But I don't know to convert a qmd to a latex file which will be included in the document.

The quarto-thesis project allow users to use their own latex files which are included in the right place. But ideally, the user should be able to do edit the frontmatter with Quarto markdown.

Do I need to create a Lua filter for that? As I am not used to Lua, I would like to check if I am not missing something simpler.

2. Can I use detect to check if a file is in the repo, do something, and if not, do something else?
   For example, if there is a cover.pdf file, I will use it as the tile page, instead of rendering a title page.

[dragonstyle]

user input: the only way I know to get data from the user and inserting in the document is by the use of metadata. This is not ideal. Instead of inserting an epigraph (for example) in the yaml, it would be better if the user just edited my frontmatter/epigraph.qmdfile and I insert that in the latex template (similar to what happens with chapters). But I don't know to convert a qmd to a latex file which will be included in the document.

You could also consider using an approach like a filter that targets a div with a particular class or id. For example, you could write a filter that finds a div with id epigraph and then capture that and create the epigraph metadata used by the template (or it could inject latex directly into the header or before body). Here is a filter that does this with abstract headings that could use as inspiration:

https://github.com/pandoc-ext/abstract-section

The quarto-thesis project allow users to use their own latex files which are included in the right place. But ideally, the user should be able to do edit the frontmatter with Quarto markdown. Do I need to create a Lua filter for that?

I would encourage you to enable the user to not only produce a LaTeX based these, but an HTML format of the the thesis as well, so the more that you can avoid the user author LaTeX directly (instead having them author markdown and then converting that to LaTeX when you need to, via filter). If you can provide a more specific example of the LaTeX front matter files I could perhaps shed some light on some approaches (not totally sure I understand the question). There are metadata points that allow including LaTeX at various places in the document, which perhaps are useful? https://quarto.org/docs/output-formats/pdf-basics.html#latex-includes

Can I use detect to check if a file is in the repo, do something, and if not, do something else?

I think your best bet here is to write a filter that inspects the file system and generates the appropriate metadata which is used by the template (e.g. it could set a metadata value indicating the cover page PDF and if the template sees that, it uses that as the cover page and doesn't emit the default title page). Does that make sense?

Given the complexity of what you're doing, I think LUA filters are likely going to be a part of the ultimate solution - we have recently really revamped the document and feature set for LUA, so I think it would be worth updating the pre-release and given this a spin (perhaps using VS Code, which will have nice LUA code completion, linting, etc.. which really make the authoring experience a lot better). https://quarto.org/docs/extensions/lua.html

[fredguth]

I created a repo with a Quarto project type of a Tufte Book layout

• I created comments with "Q:" for all questions I have.
• I still haven't created the filter like abstract-section but I understood how it works.
• I am having problems with marginnote definition because the tufte-book class also defines '\marginnote'

[fredguth]

Still a work in progress. Deployed an example to GH pages.

repo

[mcanouil]

@fredguth I added your repository as a WIP suggestion on the Awesome Quarto repository: mcanouil/awesome-quarto#163
Feel free to drop a comment when you think it's ready 😉

[fredguth]

@cderv, @dragonstyle, @rgaiacs
Since posting this question I have a deeper understanding of Quarto and how to build an extension. But there still one thing I don't know how to do: create a Lua filter to pre-process qmd files to tex and include the tex files in before-body partial front-matter (see: #2872).

The Lua filter options I saw focus on elements or blocks. I have no idea on how the book project type is built.

Alternatively, I could come-up with a templating system like quarto_titlepages, but I think this would miss the opportunity of using the flexibility of Quarto authoring features.

Any suggestion on how should I proceed? Is there an example of a filter or other kind of extension that include whole tex files in partials? Should I forget this strategy for something like what Titlepages does?

[eeholmes]

@fredguth Whenever I think "pre-process", I think makefile. I realize that's abandoning having it all bundled in quarto render but it's relatively easy to create a makefile to first process qmd files to tex and then do quarto render. To build, you run make in the terminal instead of quarto render.

[fredguth]

well, that is true. But I really wanted to keep quarto render.

[eeholmes]

Also it is harder than I thought. Running quarto render frontmatter/preface.qmd adds all the header tex stuff to the file. So it is not a matter of pre-rendering and then putting the generated tex file into include-before-body: . That's why I went with tex files wo headers for including static tex files with quarto_titlepages. https://nmfs-opensci.github.io/quarto_titlepages/06-static-files.html

[eeholmes]

In the spirit of ugly hacks, I give you this. Sometimes awful hacks can help figure out better hacks...

Main qmd. Note I had to set the engine to knitr because {bash} was trigging python engine. Quarto docs said to explicitly set engine to knitr to stop this. The bash code is rendering the front.qmd into tex. It is fortuitous that the bash code is run before pandoc adds the front.tex so the front.tex file is there when it is needed.

---
title: "test"
engine: knitr
format: 
  pdf:
    include-before-body: front.tex
---

```{bash include=FALSE}
quarto render front.qmd
```

This is the main body.

front.qmd

---
title: "test"
format: 
  latex:
    template: "default.latex"
---

This is the front matter.

default.latex; this strips off the latex header and footer so only body is in the file.

$body$

Produces