Quarto book: HELP WANTED

[friendly]

Description

[Env: Windows 10, Quarto 1.5.53]

I'm not sure this is an appropriate place to make this request, but I can't think of a better one. If inappropriate and you have an idea, please let me know.

I'm working on a new book, Visualizing Multivariate Data and Models in R using Quarto to be published by CRC Press and am running into a considerable number of problems trying to produce a LaTeX/PDF version, no less customize this to my desired.

I'm looking for someone I can hire on a short contract basis to help with this because this is more help than I can ask from the wonderful staff who maintain this discussion list.

For background, I wrote a previous book Discrete Data Analysis with R; (see the online version)
using .Rnw format, where I was able to customize the CRC Press krantz.cls style considerably to achieve a nice looking book, that I could supply in camera-ready format, complete with visual tables of contents and indices for Subject, Author, Examples.

In the new book, using Quarto, I can easily produce an HTML version and publish this to a GitHub site, but I can't even get a LaTeX/PDF version to build reliably.

Problems I need help with

• I'm using Windows 10, where MikTeX was always my LaTeX system. I was able to install LaTeX packages into a localtexmf directory and be prompted by MikTeX to install/update packages.

Quarto doesn't seem to support MikTeX for this purpose, but prefers tinytex based on the TexLive distribution. A main problem I've run into is that the TexLive package manager, tlmgr doesn't work on my system, so latex packages needed cannot be installed, and the PDF compilation fails.

• I have ideas for the book design in LaTeX that go beyond what is provided by the default krantz.cls style. These include extensive use of color in the text, custom section headers and callouts, ...

• HTML and PDF output: CRC Press allows me to have an online version of the book and there I can include, e.g., animated GIF images that would not be part of the printed version.
  I need some help with setting things up so that some text and figures can be done differently in the HTML and PDF versions.

I'm estimating that solving the TexLive problem would be short, but I'd like some help with the other problems, perhaps 10-20 hours.

[mcanouil]

Quick thought about the environment issue: have you considered using Windows Subsystem Linux (WSL) or Devcontainer (Docker)? Mostly via Visual Studio Code which natively supports those environments. (Positron might work as well but I did not try it yet)

For the HTML/PDF, conditional contents is there for that.
Depending on the complexity, you might generate all the different components in the code cells, then use conditional contents to include the components previously generated.

[mcanouil]

I forgot to mention: I'm available (it depends mostly on the amount of time required) for hire as a contractor but I'm not a LaTeX expert.

[friendly]

Thanks for the offer. Could we chat about this via email? I'm mailto:[email protected]

[mcanouil]

Sorry, I missed that message.
If you still need extensive help thus few hours of time, you can email me on my public email that you can find on my personal site.

[friendly]

Thx for the reply. On the environment: it is way too complex for me to consider such a fundamental change to my work. I'm happy with RStudio (2024.09.0+375).

I do know about HTML/PDF conditional contents and have used these in some places. But I've got to get LaTeX working with this project.

[mcanouil]

I do know about HTML/PDF conditional contents and have used these in some places. But I've got to get LaTeX working with this project.

Not sure to understand the issue.
Maybe you could share a more concrete simple use case for this?

On the environment: it is way too complex for me to consider such a fundamental change to my work. I'm happy with RStudio (2024.09.0+375).

Switching to VSCode or Positron (plus using WSL/Devcontainer) is indeed not a small leap to make.
It has advantages considering current state of the technology (AI, containers, etc.) but still a learning curve.

Link to Positron in case you are curious about the new IDE made by Posit: https://github.com/posit-dev/positron

[cderv]

Hi @friendly !

I'm looking for someone I can hire on a short contract basis to help with this because this is more help than I can ask from the wonderful staff who maintain this discussion list.

I remember when writing a book with them, we could ask some TeX question their help desk. At the time they had some issue in the .cls file provided that needed to be solved (but it was written with bookdown not Quarto). Though we needed to ask them .tex question and not Quarto question.

Looking for help to hire, I am not sure the Quarto forum as a place for such thing to be visible; So I hope you'll find some help but you should also try to ask elsewhere too.

using .Rnw format, where I was able to customize the CRC Press krantz.cls style considerably to achieve a nice looking book, that I could supply in camera-ready format, complete with visual tables of contents and indices for Subject, Author, Examples.

krantz.cls still need to be used with Quarto. So you should be able to do the same. You could also with Quarto, if you are more at ease with it, replace all partials and even the whole template. Doc is a bit hidden here: https://quarto.org/docs/journals/templates.html#latex-partials
Source file we use are here: https://github.com/quarto-dev/quarto-cli/tree/main/src/resources/formats/pdf/pandoc with the template used by Quarto here: https://github.com/quarto-dev/quarto-cli/blob/main/src/resources/formats/pdf/pandoc/template.tex

This is the one that will be used to create the final .tex and load different partials (e.g. $doc-class.tex()$ means to load in template doc-class.tex file.)
You can make a copy of this whole template and tweak, or just provides customized partials. Then configure format: pdf to use them

I'm using Windows 10, where MikTeX was always my LaTeX system. I was able to install LaTeX packages into a localtexmf directory and be prompted by MikTeX to install/update packages.

Yes - if you let Quarto build the PDF for you, then it is not possible to let MikTeX prompt you as a user. Quarto + TinyTeX does have support for automatic package installation when missing, but it does not prompt. I don't know if you can configure MikTeX that way.

If you are confortable with LaTeX compilation, and you wan't to do that yourself, you should also be able to let Quarto produce a .tex file only (format: latex), and then you can do your PDF rendering manually in a following step that you have full control over.

However, just to be clear, MikTeX is supported. Just not the prompt to user for missing packages. So all required packages should be available beforehand when you'll run a quarto render
But you don't have to switch to TeX Live if you don't want too.

A main problem I've run into is that the TexLive package manager, tlmgr doesn't work on my system, so latex packages needed cannot be installed, and the PDF compilation fails.

If you want help for TinyTeX to run on your system, please do open a discussion at https://github.com/rstudio/tinytex and I'll be happy to help

I have ideas for the book design in LaTeX that go beyond what is provided by the default krantz.cls style. These include extensive use of color in the text, custom section headers and callouts, ...

I am no LaTeX expert here, so I am not sure where those modification should be done. I have already mentioned some options above, but this make me thing also about what you can do inside the document.

• You can using Raw Tex block if you need - this will just have to be handled with your HTML version, probably with conditional as mentioned.
• We have an extension so that you can use Markdown syntax to create LaTeX environment for what you need (https://github.com/quarto-ext/latex-environment/). This could allow you do advanced LaTeX while keeping a syntax that you could also handle with HTML format (div or span with classes would be created.)

HTML and PDF output: CRC Press allows me to have an online version of the book and there I can include, e.g., animated GIF images that would not be part of the printed version.
I need some help with setting things up so that some text and figures can be done differently in the HTML and PDF versions.

As mentioned conditional content is helpful here to remove content in some output format and not others.

If you are using knitr engine to create some figures or HTML rendering, know that it should handle non HTML content by trying to make a screenshot of the interactive figure using webshot2 or webshot R package. There is also feature to condition your computation based on output format

• https://bookdown.org/yihui/rmarkdown-cookbook/html-widgets.html
• https://bookdown.org/yihui/rmarkdown-cookbook/latex-html.html

I hope this helps. And I wish you to find the hire you are looking for your book. Thanks for using Quarto !!

[friendly]

@cderv
This has been SUPER helpful. Nowhere else have I found detailed info on using Miktex vs TexLive on Windows, and in particular that Miktex is supported, but not the prompt to install missing tex packages. That can be changed in the Miktex console with the option "Prompt for missing package" -- NEVER. (This was never a problem for me before Quarto, when I used knitr, rmarkdown.)

Another gotcha was that installing tinytex did not seem to install the tlmgr package manager on my system. Not sure why, but I suppose a solution would be to [install TexLive directly] (https://www.tug.org/texlive/windows.html).

Yet another is that one cannot easily have two TeX systems on Windows, but it is possible to switch between them by changing their order in the system PATH, as described in this StackExchange question.

It would be great to have a public document on this problem, so I can start a discussion on https://github.com/rstudio/tinytex

[cderv]

Another gotcha was that installing tinytex did not seem to install the tlmgr package manager on my system. Not sure why, but I suppose a solution would be to [install TexLive directly] (tug.org/texlive/windows.html).

This should not happen. It is either an issue on TinyTeX installer or a documentation problem if you did not managed to follow the step.

Yet another is that one cannot easily have two TeX systems on Windows, but it is possible to switch between them by changing their order in the system PATH, as described in this StackExchange question.

It is possible to have different TeX system on Windows. quarto install tinytex does not set the TinyTeX it will install on PATH for that purpose. See https://quarto.org/docs/output-formats/pdf-engine.html#installing-tex

So you could have a TinyTeX for quarto and MikTeX for other usage. if you want to.

It would be great to have a public document on this problem, so I can start a discussion on rstudio/tinytex

Not sure what you meant here... sorry.

[friendly]

Progress report:

I was able to get MikTeX working to compile my book, but it was quite tedious. For the record (or anyone else with this problem):

• Re-install (the latest) MikTex

  • For my user only, NOT for all users: installs to C:\Users\friendly\AppData\Local\Programs
  • Specify YES to "Install missing packages on the fly?"
  • In the MikTeX Console, under Settings -> Directories, add my localtexmf/ to the TEXKMF directories to be searched for .sty, .bib ,,, files by kpsewhich

• In _quarto.yml turn off tinytex, and use include-* files modified from https://github.com/bgreenwell/quarto-crc. Make sure to keep-tex: true

  pdf:
    documentclass: krantz
    classoption: [10pt, krantz2]
    include-in-header: latex/preamble.tex
    include-before-body: latex/before-body.tex
    include-after-body: latex/after-body.tex
    keep-tex: true
    latex-tinytex: false
    geometry:
      - top=20mm
      - left=25mm
    code-block-bg: 'E8FFFF'  #'#f1f1f1'

• In RStudio, from Build -> Render Book -> PDF Format I get a ton of WARNINGs ([WARNING] Macro '\mat' already defined, ignoring at line 4556 column 1) but then compilation fails at the first latex ERROR

Rendering PDF
running xelatex - 1
  This is XeTeX, Version 3.141592653-2.6-0.999996 (MiKTeX 24.4) (preloaded format=xelatex.fmt)
   restricted \write18 enabled.
  entering extended mode
  
updating tlmgr

updating existing packages
ERROR: 
compilation failed- error
Package xcolor Error: Undefined color `darkgreen'.

See the xcolor package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              

• Open the index.tex file in TeXStudio.
  • Fix the LaTeX error (darkgreen is not defined in the xcolor package. I fix that using \definecolor{darkgreen}{RGB}{1,50,32}.
  • Compile again to resolve cross-references, build the table of contents
  • Run Tools -> Index (makeindex) to build the index
  • Compile again.

Questions:

Note that TeXStudio uses pdflatex.exe -synctex=1 -interaction=nonstopmode "index".tex so it doesn't stop at the first error. Is there any way to use nonstopmode in the RStudio compile?

Clearly, I need a Makefile for this, but I'm not sure how to specify one that I can run from RStudio.

How can I change the name of the output files from index.tex, index.pdf, ...? I tried this, to no effect:

book:
  title: "Visualizing Multivariate Data and Models in R"
  output-file: "Vis-MLM"

[cderv]

Note that TeXStudio uses pdflatex.exe -synctex=1 -interaction=nonstopmode "index".tex so it doesn't stop at the first error. Is there any way to use nonstopmode in the RStudio compile?

Do you mean when doing In RStudio, from Build -> Render Book -> PDF Format ?

if you use RStudio to build to PDF through Quarto, this is really Quarto calling TeX engine for you.
Quarto has some option your can set for the engine - See https://quarto.org/docs/output-formats/pdf-engine.html#alternate-pdf-engines

So you could pass any flag you want.

Just so you know, we did our best to make the LaTeX PDF experience as smooth as possible thanks to TinyTeX integration. When you opt out of it, it will be more advanced and manual, but I believe we offer entry point for configuration.

You shouldn't have to open a index.tex in TeXStudio to have a PDF render from a Quarto project really.

Happy to work with you to make your project work with Quarto.

[friendly]

Thanks @cderv for continuing to follow up, & for the link to the PDF-engine docs. I'm glad your team is working toward more seamless LaTeX/PDF integration and will give TinyTeX another whirl when I get a chance.

I want to express my general thanks to all the Quarto-dev folks who respond helpfully to many, many queries.

[vincentarelbundock]

@friendly I am also writing a book for CRC and have gotten the PDF to build. It was quite tricky, dealing with lots of "partials", but it seems to work OK for now.

I'm happy to share the files and chat a bit if you think that would be helpful.

You can reach me at my Université de Montréal email, easily findable on the web.

[friendly]

Thanks so much, Vincent. I'll contact you by email.

[baptiste]

I also got their template to work last year despite obnoxious incompatibility with hyperref (and useless tech support). However, with Taylor and Francis' recent "strategy" to surreptitiously sell out the work of their authors "for the benefit of LLMs", I am considering ditching them and sticking with my online version, and publishing elsewhere the pdf with a template that actually looks good (theirs is too narrow for good multi-panel figures).

https://pivot-to-ai.com/2024/07/20/taylor-francis-feeds-academics-work-to-microsoft-ai/

(sorry for the sidenote rant, but I wanted to share my experience with their Latex "support" and now this pathetic development)

[friendly]

For the record: most of my problems are now SOLVED, in the sense that I can reliably render pdf and make it all the way to producing a PDF version without errors (but it takes ~6 min to build!)

I'm documenting this here in case others find this useful.

• switched to tinytex. It took me a long time to figure out how to do this on my Windows 10 system. One key was to use tinytex::install_tinytex(bundle="TinyTeX") to get a fuller tex installation.
• Read & re-read the docs on https://quarto.org/docs/output-formats/pdf-engine.html#alternate-pdf-engines and set a bunch of options

  pdf:
    documentclass: krantz
    classoption: [10pt, krantz2]
    include-in-header: latex/preamble.tex
    include-before-body: latex/before-body.tex
    include-after-body: latex/after-body.tex
    keep-tex: true
    latex-tinytex: true
    # can also use pdf-engine: LatexMk / pdflatex ...
    pdf-engine: xelatex
    pdf-engine-opts:
      - '-interaction=nonstopmode'
    latex-makeindex: makeindex
    latex-clean: false
    geometry:
      - top=20mm
      - left=25mm
    code-block-bg: "dce8e7" # or #e6f2ee, not 'E8FFFF'  #'#f1f1f1'

• modelsummary() was generating mysterious errors that halted the quarto render process. This problem and my (hacky) solution is described in https://github.com/vincentarelbundock/modelsummary/issues/901and modelsummary pdf and latex issues friendly/Vis-MLM-book#36

• INDEXING: Figured out to automatically have every reference to a {package} and any dataset reference of the form pkg::dataset generate index entries.

  • This involves using inline function calls like r pkg("lattice") in the text rather than just lattice
  • similarly, use r dataset("carData::Prestige") generates index entries under datasets|Prestige and Prestige data. Some code for this is in this gist
  • Also defined indexing macros in preamble.tex for manual indexing

\newcommand{\IX}[1]{\index{#1}#1}
\newcommand{\ix}[1]{\index{#1}}
\newcommand{\ixmain}[1]{\index{#1|textbf}}
\newcommand{\ixon}[1]{\index{#1|(}}
\newcommand{\ixoff}[1]{\index{#1|)}}
...

• Lots of custom LaTeX macros are defined in a file latex-command.qmd for simplifying math notation, e.g.,

\renewcommand*{\vec}[1]{\mathbf{#1}}
\providecommand{\trans}{^\mathsf{T}}
\providecommand*{\mat}[1]{\mathbf{#1}}
\providecommand*{\diag}[1]{\mathrm{diag}\, #1}
\renewcommand*{\det}[1]{\mathrm{det}(#1)}
\providecommand*{\rank}[1]{\mathrm{rank} (\mathbf{#1})}
\providecommand*{\trace}[1]{\mathrm{tr} (\mathbf{#1})}
\providecommand*{\dev}[1]{(#1 - \bar{#1})}
\providecommand*{\inv}[1]{\mat{#1}^{-1}}
\providecommand*{\half}[1]{\mat{#1}^{1/2}}
\providecommand*{\invhalf}[1]{\mat{#1}^{-1/2}}

• This is 'sourced' at the beginning of each chapter, because HTML build treats each chapter separately.

::: {.content-visible unless-format="pdf"}
{{< include latex/latex-commands.qmd >}}
:::

• Note the use of \providecommand* rather than \newcommand* to avoid WARNINGs \rank already defined from the PDF build

Still TODO

• For creating the index with makeindex, I'd like to use my own book.ist style file, but I can't find out how to include options for makeindex, In a Makefile I would use:

index :
#  subject index
	makeindex -o book.ind -s book.ist book.idx

My book.ist style file prints a large capital letter for each section

heading_prefix   "{\\bfseries\\sffamily\\fbox{ "    %% insert in front of letter
heading_suffix   " }\\hrulefill}\\nopagebreak\n"               %% after letter
headings_flag       1                               %% turn on headings
indent_length       10         % default: 16

• I'd also like to create an author index automatically from the citations in the book. This can be done with the LaTeX authorindex package, but it comes with a perl script that must be run to actually generate the author index. In a `Makefile for an earlier book I had:

#  author/editor index -- requires authorindex script
authorindex:
	perl authorindex -d book

** But, how can I integrate this into my workflow**?

• Figure out how to use pandoc partials / templates to overcome some of the clunky features of the krantz.cls used by CRC press

• Figure out how to add a cover image to the PDF version. I have one that is a complete cover, so all I want to do is prepend it to what quarto produces. pdfjam can do this, but can't find a Windows binary.

[cderv]

switched to tinytex. It took me a long time to figure out how to do this on my Windows 10 system. One key was to use tinytex::install_tinytex(bundle="TinyTeX") to get a fuller tex installation.

quarto install tinytex is another option that we document: https://quarto.org/docs/output-formats/pdf-basics.html#prerequisites. Wasn't it working on your Windows environment?

I'd also like to create an author index automatically from the citations in the book. This can be done with the LaTeX authorindex package, but it comes with a perl script that must be run to actually generate the author index. In a `Makefile for an earlier book I had:

When is this to be run? I assume before the PDF is created from the LaTeX file?
I don't think we have an endpoint yet for running another command in the pipeline recipe.

I wonder if the easiest thing for you is to

• quarto render --to latex
• And then control the whole latex to pdf rendering process. You could call whatever tool you like, which the options you need. No need to configure Quarto LaTeX mk as you would just skip the PDF building phase by control, and you would control it totally. Did you try this approach ?

Figure out how to use pandoc partials / templates to overcome some of the clunky features of the krantz.cls used by CRC press

You are already using includes

    include-in-header: latex/preamble.tex
    include-before-body: latex/before-body.tex
    include-after-body: latex/after-body.tex

Isn't this enough ? I don't think Quarto template has anything regarding the krantz.cls that you pass as a custom documentclass.

Though, if you need to tweak some of the template content because incompatible with krantz.cls, then the template you can modify are here: https://quarto.org/docs/journals/templates.html#latex-partials

Just find the one you need to modify by looking at the .tex result you have and the partials source on GitHub. This will allow you to identify which one is involved.

Figure out how to add a cover image to the PDF version. I have one that is a complete cover, so all I want to do is prepend it to what quarto produces. pdfjam can do this, but can't find a Windows binary.

I don't know about pdfjam, but looking at their README (https://github.com/pdfjam/pdfjam#-installation), I see it should be available with TeX Live. https://ctan.org/pkg/pdfjam
It may work on windows using a bash emulation: Cygwin or else.
Or if you have WSL, this should work.

However, what pdfjam done, you can do it with other tools. R package like qpdf wraps some tooling to manipulate pdf (https://docs.ropensci.org/qpdf/). pdftk is another tooling: https://www.pdflabs.com/tools/pdftk-the-pdf-toolkit/

Otherwise, on LaTeX side only, I remember titlepic to be an option: https://github.com/ttencate/titlepic
It is a CTAN package: https://ctan.org/pkg/titlepic

Hope it helps

[mcanouil]

You can use latex-makeindex-opts to pass array of options to the program you've set with latex-makeindex.

[friendly]

Thx. It would be so good to be able to see a complete list of all the latex related options in the docs, perhaps with examples.
In my case, I think this would be

latex-makeindex-opts: -s latex/book.ist

but I'm not sure of the yaml syntax for this

[mcanouil]

Look at the references pages.
The option is listed.

As documented, the option takes an array.

[cderv]

It would be so good to be able to see a complete list of all the latex related options in the docs, perhaps with examples.

This is also documented in the main guide doc: https://quarto.org/docs/output-formats/pdf-engine.html#quarto-pdf-engine

We try to document as much as possible in the guide to give context, but the reference page is indeed the full extensive list of supported options: https://quarto.org/docs/reference/formats/pdf.html#latexmk

Adding examples for everything is difficult as it can easily complicate the main documentation. Maybe there should be a blog post or a dedicated example document for cases like yours where you do extensive configurations to tweak the default behavior. 🤔

[friendly]

@mcanouil Sorry for being dense. You say "As documented, the option takes an array", but that dosen't indicate how to specify that in my _config.yml or else there is something peculiar with how makeindex is called by Quarto.

I tried:

    pdf-engine-opts:
      - '-interaction=nonstopmode'
    latex-makeindex: makeindex
    latex-makeindex-opts:
      - "s latex/book.ist"
    latex-clean: false
   ...

But this errors with:

making index
  Input index file s latex/book.ist not found.
  Usage: makeindex [-ilqrcg] [-s sty] [-o ind] [-t log] [-p num] [idx0 idx1 ...]

I get the same result if I use -s latex/book instead:

    latex-makeindex: makeindex
    latex-makeindex-opts:
      - "-s latex/book.ist"

[cderv]

Array as YAML array as you are in a YAML file.

Try

latex-makeindex-opts: ["-s", "latex/book.ist"]

[mcanouil]

@friendly Sorry, as Christophe pointed out, the documentation and I were making reference to the YAML specification.
Arrays are sometimes called lists or sequences in this context.
For program, providing options as array means that there should not be any space separators in the values.
From there you have two syntaxes, the one showed by Christophe (compact) latex-makeindex-opts: ["-s", "latex/book.ist"] and the expanded one you might be more used to:

latex-makeindex-opts:
  - "-s"
  - "latex/book.ist"

[friendly]

Wow, I'm realizing I could market this as a multilingual book. I'm using:

• R
• tidyverse, ggplot2
• Rmarkdown
• Quarto
• YAML
• LaTeX
• ...

each with it's own syntax and structure.

Thanks guys for putting up with this. :)