Auto-copy static resources to _files subdir also for standalone HTML documents?

[floholl]

Description

I am using Quarto for teaching purposes and am hoping to establish roughly the following directory structure:

• A Quarto project (_quarto.yml) for all my courses at a given educational institution, such as to efficiently share metadata among my classes (so I don't have to duplicate data on the name of the institution, etc.)
• A subdirectory for each of my courses, with further sub-subdirectories for each class meeting
• An img subdirectory for static resources (images) that multiple courses should be able to draw from (to avoid having to maintain multiple copies of the same figures, for example).

For each class meeting in any given course, I would like to be able to generate

• a revealjs presentation
• a stand-alone html document as a handout for post-class reference

Note that in this scenario, the generated materials (i.e., the .html file plus _files directory) for each class meeting should function completely independently as a stand-alone document. In other words, I am not (for now, at least) aiming to collect them into a larger book or website project. Rather, I would like to be able to upload the .html file plus _files directory for a given class meeting to a dedicated (and independent) folder on a web server, open the .html file there, and everything should work, even if I did this for just a single class meeting.

The problem that I am facing is how to efficiently include images from the Quarto project's shared img directory in the .qmd files for the individual class meetings. I can specify the path to these images in relative terms (since they live within the Quarto project), and on my local machine, the images will of course be recognized. However, as soon as I take the generated materials for a single class meeting out of this context - such as by uploading them to a web server in isolation as described above - the image will of course no longer be found.

Of course I could upload the image folder to that web server and try to mimic my local file tree structure there, but this would negate the idea of complete independence of individual class meetings (which is non-negotiable for my intentions). Rather, it would be great to automatically copy the required static resources from the project-wide img directory to the _files directory. I understand this is possible for Quarto website projects, but note again that I am trying to build not a full website (in the Quarto sense of the term) from my Quarto project, but rather independent HTML documents. Do I understand correctly that the same copy feature is not available for standalone HTML documents? If so, would it perhaps worth considering this as an option, or are there possible workarounds?

(My current workaround is to maintain a separate img subdirectory in each class meetings directory, which contains symbolic links to the requierd image files in the project-wide img directory. When I then copy the files to the web server, I can automate resolving those symbolic links to copies of the actual files. However, this is inconvenient, as I now need to specify the location of each image twice, once in the .qmd itself, and once as a symbolic link.)

[mcanouil]

Why not use embed-resources?

I believe you mean "single document rendering" instead of "standalone" which has a very specific meaning.

Could you try in the future to share a bit of code to show/reproduce concretely your setup?

[mcanouil]

Also, the documentation you've linked about resources works inside any project, including default.

• document.qmd

---
title: "Reproducible Quarto Document"
format: html
engine: markdown
resources:
  - whatever.png
---

![image](/whatever.png)

• _quarto.yml

project:
  type: default
  output-dir: _output

quarto render document.qmd

[floholl]

@mcanouil, thanks for your suggestions!

Actually, I wasn't aiming at single-document rendering (but rendering to an index.html plus accompanying _files directory), out of a (perhaps unjustified?) concern that single-document rendering "might not work well" (admittedly vague) for large videos, for example.

This is also why I hadn't further explored embed-resources in this context, but I'll test it again now with some video and audio files. I also recall noticing that the documentation for embed-resources mentions "linked scripts, stylesheets, images, and videos", but not audio files (which I also need) - I'm guessing that data: URIs also work for audio, though?

Yes, sorry, for not providing a code example, which in this case is due to the complexity of even an MWE's project structure, along with the dependence on external files. I'll see whether I can come up with a meaningful MWE after further experiments with embed-resources.

Excited to read that the resources: syntax works for any project - I wasn't aware of this (perhaps incorrectly assuming that because I stumbled across it in the website projects documentation, it would only apply to website projects) and will give it a try, too! I guess one complication here might be that I neither want to hand-pick the resources to be included (which would again require identifying them in two locations, once in the metadata resources: field, and once in the .qmd body where the image actually appears), nor do I want to use wildcards à la *.svg, which would also include images not used by a given class meeting (but others).

[mcanouil]

I get that your use case is complex but being to make it simple helps ensure others understand exactly what you need and provide more efficient help.

Maybe embed-resources is not suitable here or maybe it is, hard to tell without anything concrete.
Now with the additional information, I think, using project resources might be better, but no guaranty there, you indeed will have to try and see what suits your project.

[floholl]

Yes, I'll do some further digging, thanks. One preliminary result that further supports your last argument:

ERROR: Reveal plugin 'RevealChalkboard is not compatible with self-contained output

[floholl]

I am still digging into this in more detail, but will probably open a new issue with reproducible code at a later stage. With regards to this issue here, the key information for me, which I am now marking as an answer, is that resources are auto-copied to the output directory not just for "website" projects, but also for "default" (and probably, in fact, any?) projects.

A few notes on @mcanouil's comment that I marked as an answer:

quarto document.qmd

was likely intended to read

quarto render document.qmd

Also, note that the resources: field in document.qmd is not strictly required in this example, precisely because whatever.qmd will be auto-detected and copied. For the record, the documentation mentions this:

If this auto-detection fails for any reason, or if you want to publish a file not explicitly linked to from within your site, you can add a resources entry to your configuration.

In other words, one can further break down this MWE to the following:

.
├── document.qmd
├── whatever.png
└── _quarto.yml

document.qmd:

---
title: "Reproducible Quarto Document"
---

![image](/whatever.png)

_quarto.yml:

project:
  output-dir: _output

Render via

quarto render document.qmd

which yields

.
├── index.qmd
├── whatever.png <=== original image
├── _output
│   ├── index_files
│   │   └── libs
│   │       ├── ...
│   ├── document.html
│   └── whatever.png <=== copied image
└── _quarto.yml

What does seem to be essential for the image copy operation to work, however, is the specification of an output-dir for the project (here: in _quarto.yml). I think this is what might have mistakenly led me to believe that the auto-copy process only works for website projects.

I'll dig into more details of this and will then likely open a new issue. In particular, I'm looking into whether the location of that output-dir can be specified in relation to document.qmd (which in my specific use case differs from the quarto project's root). For the record, I tried execute-dir: file to no avail to achieve this, and also vaguely remember reading an issue specifically on this problem, but cannot seem to find it now.

[mcanouil]

Indeed, there is no reason for Quarto to duplicate/copy assets if your output lives with your input as all paths will be correct.

You probably should read or at least be aware of:

• Output directory in quarto-cli #2171

[floholl]

Thanks for this helpful comment, which allows me to articulate in more detail now what my problem is.

MWE 1: Trivial case

Your first comment, if I am not mistaken, describes - for example - the following situation:

.
├── document.qmd
├── whatever.png
└── _quarto.yml

document.qmd:

---
title: "Reproducible Quarto Document"
---

![image](/whatever.png)

_quarto.yml, but this time without output-dir:

project:
  type: default

and

quarto render document.qmd

I understand that in this case, whatever.png is already in the right place, hence no need to copy it:

.
├── document_files
│   └──...
├── document.html
├── document.qmd
├── Makefile
├── whatever.png
└── _quarto.yml

My actual situation: Subdirectory for a class session

However, my actual situation breaks down to the following (apologies for file renames, but should be clear enough):

.
├── img
│   └── my.png
├── _quarto.yml
└── session1
    ├── index.qmd

where session1/index.qmd:

---
title: "Reproducible Quarto Document"
---

![image](/img/my.png)

and with _quarto.yml:

project:
  output-dir: _output

I then compile (and this will probabyl be important for next example) from the subdirectory:

cd session1
quarto render index.qmd

which then creates, as expected, _output relative to _quarto.yml:

.
├── img
│   └── my.png
├── _output
│   ├── img
│   │   └── my.png
│   └── session1
│       ├── index_files
│       │   └── ...
│       └── index.html
├── _quarto.yml
└── session1
    ├── index_files
    ├── index.qmd
    └── Makefile

This all works great.

My goal: Keep all output in class session subdir, including resources such as images

My goal, however, is to create all output inside the session1 subdirectory from which I run quarto render, since that's where I tend to cd to and then do my work (vim + Terminal). However, if just remove the output-dir option by changing _quarto.yml in the above example to:

project:
  type: default

then I get

.
├── img
│   └── my.png
├── _quarto.yml
└── session1
    ├── index_files
    │   └── ...
    ├── index.html
    ├── index.qmd

where index.html links to ../img/my.png. No problem there, until I then take the session1 folder out of its Quarto project context and upload it to a web server (without the img folder it lives "next to").

In this context, thanks for the link to issue #2171, which I am pretty sure was the one I vaguely remember (as I mentioned in my last post). If I am not mistaken, you following comment there describes precisely this situation:

Note that trying to render from a sub-directory of a Quarto project will still "fail"

What I need to investigate next: Does it help to run my quarto render commands exclusively from the Quarto project's root directory? Or might there be another way to keep input and output "together" while making the output "autonomous" and still sharing project resources?

Also, I noticed in issue #2171 a reference to this comprehensive list which feels relevant to my use case, but which I first need to identify there (suggestions are welcome). :)

[mcanouil]

Maybe you would be better off using a post-render script in combination with output-dir.
Note that you can set output-dir from the command line, so you could have some kind of make file or whatever.

You could also use profiles, like one profile per sub-directory but of course located at the parent level.

Anyhow, what you want is not possible in Quarto for designed reasons.

• output-dir uses the root of a project and is for project. So without profiles or using command line flag, you won't have the output generated in a sub-directory if you are rendering a sub-directory input.
• if you set each sub-directory as a project, then your resources would be outside of the projects so you wouldn't be able to use it as it's against the design of Quarto for reproducibility.

[floholl]

Maybe you would be better off using a post-render script in combination with output-dir.

Yes, that's where my thinking has also been heading...

Anyhow, what you want is not possible in Quarto [...]

Thanks for confirming that.