Executing a Quarto project (located in a subdirectory) from the root of an R project

[arnaudgallou]

Description

I'm using a research compendium for a project. The compendium has the basic structure of an R package plus some extra directories for the analyses and manuscript. I'm using a .Rprofile to load helper functions and data required for both the analyses and manuscript.

A simplified structure of the project looks like this:

.
├── .Rprofile
├── DESCRIPTION
├── NAMESPACE
├── R
│   └── utils.R
├── data
│   └── data.parquet
├── man
├── manuscript
│   ├── references.bib
│   └── index.qmd
├── foo.Rproj
└── scripts
    ├── analyses.R
    └── results.R

Everything is interconnected. E.g. index.qmd uses code in results.R sourced with knitr::read_chunk() and <<>>. The code in results.R uses helpers and data below R/ and data/, respectively, which are loaded via .Rprofile.

I render index.qmd using

quarto::quarto_render(
  input = "manuscript/index.qmd",
  execute_dir = "."
)

This works because .Rprofile is executed before rendering index.qmd and all the paths are relative to the root of the R project.

I'm now trying to replace this standalone .qmd file with a Quarto manuscript project but I'm running into executing environment and path issues because everything is now relative to the Quarto project. Using the same quarto::quarto_render() call as above no longer works because .Rprofile isn't executed and the data now have wrong paths.

My question is: is there a way to make a Quarto project work with that workflow?

The similar discussions I've found seem to imply there's no ways to make it work but since my workflow is quite different I prefer asking for confirmation.

Edit

Here's a real life example.

[mcanouil]

You can change the execute directory via Quarto CLI: https://quarto.org/docs/projects/code-execution.html#working-dir

For related R package development, the repository is https://github.com/quarto-dev/quarto-r

Note that the R package is a limited wrapper to Quarto CLI.

[mcanouil]

A project is self-contained, if your ms_project contains a _quarto.yml, then the root is where _quarto.yml, thus no .Rprofile there etc. That's by design.

[arnaudgallou]

That's what I read in other posts when people were trying to run Quarto projects from the command line. Quarto projects don't seem to be completely self-contained when using quarto::render(execute_dir = ".") though and that's confusing.

For example the read_chunk() here is able to read the file correctly despite that the path is relative to the R project directory, not the Quarto project directory. Unless quarto::render(execute_dir = ".") only applies to code chunks (?).

I'll use the pre-render and post-render options to copy the .Rprofile to the Quarto project directory and delete it when the rendering is complete for now.

There are certainly good reasons for this design but it's a pity there's no better integration of Quarto projects with R projects. I could of course have the Quarto project at the root of the R project but that would make things less organized.

[mcanouil]

It's not about setting execute-dir, it's about the explicit _quarto.yml or not which defines the root of a project.

So, it is self-contained, the thing is that you basically are changing the root explicitly or implicitly for single-file (where your start Quarto).

Note also that quarto::render(execute_dir = ".") is not Quarto. It's a wrapper around Quarto.
Meaning you are adding another layer on top of Quarto, in addition to R which might have a working directory different from the one you might expect.

I do understand the appeal coming from R to do everything in R, but when doing tests, etc., I would really recommend you use Quarto CLI directly and set the right options quarto render --help.

[cderv]

I'll try to clarify a few things

Quarto projects don't seem to be completely self-contained when using quarto::render(execute_dir = ".") though and that's confusing.

execute_dir = "." in the R function is equivalent to calling quarto render <file> --execute-dir .. Quarto will normalize the path . from where the quarto render command is called. So in here,

quarto::quarto_render(
  input = "manuscript/index.qmd",
  execute_dir = "."
)

it will be from root of you RStudio project. If you call that from another folder it would be different. Using . is not the safest to insure correct path is used.

For knitr engine, --execute-dir will set the value for knit_root_dir argument in rmarkdown::render(), which is setting knitr::opts_knit$set(root.dir = ). So it really controls what R engine is doing.

You can change the execute directory via Quarto CLI: quarto.org/docs/projects/code-execution.html#working-dir
(...)
Unless I'm missing something, this will set the working directory to the Quarto project, not the R one.

If no --execute-dir is passed at command line, then execute-dir option in project YAML will be used to set the value to the project path when execute-dir: project. Otherwise, the directory of the source file being rendered is used.

After a bit more testing, it seems that the only issue is with .Rprofile. If I copy the .Rprofile into ms_project/, the document will render properly. That means the executing directory is the root of the R project in the end. Although it's executed, the .Rprofile at the root of the R project seems to have no effect.

I'll check with your exemple but here is how it is supposed to work.

Quarto will call R from working dir where quarto render is called, unless there is a project, and in which case the project dir will be used.

In your case, the project is inside ms_project. So when you call quarto render ms_project/index.qmd, it will find this is a project and so call R from there. An .Rprofile there will be loaded.
Doing quarto render ms_standalone/index.qmd is a single file render. In this case, there is no project so the working dir where quarto render is called will be used, i.e. your project root.

I'll use the pre-render and post-render options to copy the .Rprofile to the Quarto project directory and delete it when the rendering is complete for now.

This seems like a good workflow for your compendium organization. You could also see if having a .Rprofile in your subfolder can source a script in the root folder.

Other comment about .Rprofile. The file used can be specified by R_PROFILE_USER environment variable. (from man page). So maybe there is way to tell Quarto to always load a specific one no matter where is called. You would have to deal with path resolution to make that safe.

A related idea for you specific project organization. Quarto project inside your compendium overall structure does not know it is part of such compendium structure. You may set some environment variable or other config to tell Quarto project absolute path to your compendium root maybe this could be useful. Or at least for R code you could leverage here package probably.

Quite long answer, but I hope this will help make your workflow more understandable regarding Quarto's behavior.

[arnaudgallou]

Thanks a lot for the detailed answers. That clarifies many things.

You could also see if having a .Rprofile in your subfolder can source a script in the root folder.

The problem is that I also need the .Rprofile to initialize the overall project, in particular to load helper functions and the required libraries to run both the files below scripts/ and the manuscript (which in turn depends on files below scripts/ and R/). So it has to be run at the root of the R project too.

The file used can be specified by R_PROFILE_USER environment variable. (from man page). So maybe there is way to tell Quarto to always load a specific one no matter where is called. You would have to deal with path resolution to make that safe.

Quarto project inside your compendium overall structure does not know it is part of such compendium structure. You may set some environment variable or other config to tell Quarto project absolute path to your compendium root maybe this could be useful.

These are certainly ideas I need to try out.

Or at least for R code you could leverage here package probably.

I'm not a big fan of this package to be honest. It certainly has its use in some situations but since R projects already handle directories and paths, I feel like using here in that case is redundant and add complexity to the overall project.

I hope this will help make your workflow more understandable regarding Quarto's behavior.

It certainly does. In fact I'm currently experimenting that type of compendiums (with a some extra perks) for an actual research and that works so well at the moment that I'm planning on making a package out of it when I've time. This .Rprofile thing was the main issue that stopped me. I now have all the information I need to find out the best solution.

[cderv]

I feel like using here in that case is redundant and add complexity to the overall project

The feature that allows you to define a specific meaning for what is root directory is to me a great feature, especially in complex project structure. It could be here or the low level rprojroot directly.

xfun::proj_root() also allows this without all the features around here.