Can't control the render order on GitHub Actions ubuntu-latest

[eculler]

Bug description

I am having this problem as well on GitHub Actions with the ubuntu-latest runner, despite:

• listing files individually in project/render and
• Numbering the files so that alphanumeric order should be the correct one

The render runs as expected on my system (MacOS Sonoma 14.5).

I need the notebooks to render in order, because they use jupyter storemagic to carry variables over from one to the next.

Steps to reproduce

My project has the following in _quarto.yml:

project:
  render:
    - pages/03-git-github/02-github-collaboration/02-pull-requests.qmd
    - notebooks/01-climate/climate-31.qmd
    - notebooks/01-climate/climate-32.qmd
    - notebooks/01-climate/climate-33.qmd
    - notebooks/01-climate/climate-34.qmd

Expected behavior

I expect the files to be run in the order they appear in the render list, or the order they appear in the table of contents, or in alphabetic order. As it is, the run order seems to be entirely unpredictable and depends on the system.

Actual behavior

the actual run order in my project on GitHub Actions ubuntu-latest runner is:
- notebooks/01-climate/climate-31.qmd
- pages/03-git-github/02-github-collaboration/02-pull-requests.qmd
- notebooks/01-climate/climate-33.qmd
- notebooks/01-climate/climate-32.qmd
- notebooks/01-climate/climate-34.qmd

Your environment

• GitHub Actions ubuntu-latest runner
• quarto installed with quarto-actions/setupv2 in an activated conda environment

Quarto check output

Run quarto check

Quarto 1.4.557
[✓] Checking versions of quarto binary dependencies...
Pandoc version 3.1.12: OK
Dart Sass version 1.58.3: OK
Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
Version: 1.4.557
Path: /usr/share/miniconda/envs/learning-portal/bin

[✓] Checking tools....................OK
TinyTeX: (not installed)
Chromium: (not installed)

[✓] Checking LaTeX....................OK
Tex: (not detected)
(|) Checking basic markdown render....
[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
Version: 3.10.14 (Conda)
Path: /usr/share/miniconda/envs/learning-portal/bin/python
Jupyter: 5.7.2
Kernels: ir, python3
(|) Checking Jupyter engine render....
[✓] Checking Jupyter engine render....OK
(|) Checking R installation...........
[✓] Checking R installation...........OK
Version: 4.3.3
Path: /usr/share/miniconda/envs/learning-portal/lib/R
LibPaths:
- /usr/share/miniconda/envs/learning-portal/lib/R/library
knitr: 1.48
rmarkdown: 2.27
(|) Checking Knitr engine render......
[✓] Checking Knitr engine render......OK

[mcanouil]

• Control order of rendering #6944

Tracked:

• Epic: Execution Planner #6518

[allefeld]

This is a duplicate of #10490.

@mcanouil: Yes, the official support will come with the epic execution planner, but previously listing files explicitly in project / render was a workaround. This workaround now stopped working.

[mcanouil]

I know but a workaround is a workaround.
There was/is/will be no guarantee with workarounds.

Relying on workarounds is never something to be done.

Edit: as there is a brief mention in the documentation, it's possible that the "workaround" was at some some point considered a feature which would make this a regression from previous versions.

To be clear #10490 is the duplicate not the
Other way around.

[allefeld]

The problem is that I need to be able to control the order for an existing project, and the execution planner is still in the medium to far future.

Is it a legitimate solution in the current situation (to avoid the term "workaround") to render from a shell script, where the script calls quarto render <file> for each file in sequence?
Does quarto render for a project do something different than quarto render <file> for all the render targets in the project?

[mcanouil]

To be clear, I never said it's not issue or not a bug.
I am only saying that all this comes in a more global implementation.
Now, whether or not it's a bug, that the "documented workaround" stopped working, I cannot say... thus the fact that the issue is opened and not assigned to anyone or any milestone.

There are no arguing about the need to be able to have ordered rendering.

[allefeld]

I understand.

Could you please comment on my approach to control rendering from a shell script? Should that work?

[cscheid]

Just to repeat from a slightly more official source, Quarto doesn't promise control of rendering order. The robust solution is to use a pre-render script to produce all necessary inputs before the files are rendered. We understand that this isn't ideal, but we truly can't fix this right now given the rest of our workload.

[allefeld]

@cscheid, I'm not asking you to completely overturn your work plan or take on additional work, but for a bit of practical advice for a concrete problem I need to solve.

Regarding producing the necessary inputs before – the thing is that the code which creates those inputs is in the form of qmd files, which can't be run from pre-render scripts (I think?).

What I'm currently doing is not have a pre-render script called from Quarto, but a render script calling Quarto, for all qmd files separately in order. So instead of

quarto render <folder>

I do

quarto render <folder>/<file1.qmd>
quarto render <folder>/<file2.qmd>
…

That seems to be working okay, except that I don't get sidebar contents in the main index.md rendering, but that may be an unrelated issue.

Could you please comment whether that is a reasonable approach, or whether there are any drawbacks to it?

[cscheid]

That seems to be working okay, except that I don't get sidebar contents in the main index.md rendering, but that may be an unrelated issue.

It's hard to have a discussion without a reprex around it. Can you create one first?

[allefeld]

I'll investigate the sidebar issue, and if it points to a problem with Quarto I'll create an issue incl. reprex.

But my question is about the general behavior of quarto render. Say I have a Quarto project with the following structure:

folder/
folder/quarto.yml
folder/index.md
folder/file1.qmd
folder/file2.qmd

can I expect the result of

quarto render folder/index.md
quarto render folder/file1.qmd
quarto render folder/file2.qmd

to be identical to the result of

quarto render folder

(apart of course from enforcing order)?

If yes, I then I have a usable workaround to ensure order of rendering.