Experience using tectonic to compile a large & complex document

[thangleiter]

I have recently used* tectonic to compile my PhD thesis, and thought I'd share some of my experience and take-aways for what is hopefully the most complex TeX document I will ever need to build.

*At least, and this is probably the most crucial point, up until it stopped working. I have pulled both a LuaLaTeX/TL2025 and a Tectonic/TL2022 version along in parallel, until I finally needed to compile everything together (I built only smaller parts with \includeonly{} up to that point). Then I ran into memory bounds:

! TeX capacity exceeded, sorry [main memory size=5000000].

Of course, this is a XeTeX limitation. However, in vanilla XeTeX you could modify texmf.cnf and increase the memory size (latexmk uses 50MB when compiling with XeTeX, which really does not seem a lot).
Being able to do this in tectonic, or increase the available memory by default, is thus crucial for tectonic to be a viable alternative for documents of this complexity.

In general, I think it's much more enjoyable to work on a tectonic document than a regular LaTeX document. tectonic (V2) allows for a directory structure similar to a code project. For instance, including the document class was simply a matter of including it as a git submodule and tell tectonic that's where to find additional latex files. By contrast, for this to be up-to-date and work relatively care-free, I had to write git hooks to copy/link those files to the correct TEXMF directories with plain LaTeX---the alternative being of course populating and blowing up the top-level directory with the files.

A logical extension of this concept is in my opinion allowing fonts to be shipped with the project in the same fashion (#1238).

What I found mildly cumbersome is being stuck on TL2022. I frequently found myself in the situation that some bug was fixed or feature added in versions of packages shipped with more recent TL versions. I am aware that one can build their own bundle, but hosting it is another matter, which kind of defeats the portability aspect of tectonic.

Finally, for my purposes I had to add the patches suggested by @Danacus in #704 (comment) to incorporate glossaries. Being able to add custom commands to enter the build process on a per-project basis would be quite the convenient feature, similar to how you can add perl scripts to a latexmk build, for example. Having also recently looked into Pixi, I think something like the tools table there in the Tectonic.toml would be neat -- see also #1304 (comment).

[CraftSpider]

Thank you for the in-depth review. I'm glad the Tectonic model was a good experience, even if the tool still has a ways to go.

The memory limits are, hopefully, temporary. The port to Rust work is slow, but as part of it I've been working to switch to a more classic dynamically growing memory model, instead of fixed size buffers. (Among other hopeful improvements. I'm also hoping to improve errors in V2 mode, once the framework for this is in Rust). This work has been my main focus recently, over new features.

I believe @rm-dr is more familiar with bundles than me. I thought Tectonic supported a TL2024 bundle at this point, but I would be interested in building and shipping more bundles if that's helpful. The biggest constraints there I imagine are storage based.

Finally, as a maintainer, both the local fonts and the tools idea definitely also interest me. I'm not in a position to promise delivery of them soon, but I am interested in getting them into Tectonic eventually.