Large `brand.yml` leads to `Failed to spawn 'pandoc': Argument list too long (os error 7)`

[mcanouil]

Depending on the environnement, when Brand YAML file is too large, Quarto fails to spawn Pandoc and crashed.

ERROR: Error executing '/opt/quarto/bin/tools/x86_64/pandoc': Failed to spawn '/opt/quarto/bin/tools/x86_64/pandoc': Argument list too long (os error 7)

Full log can be found at https://github.com/mcanouil/issue-large-brand-yml/actions/runs/18461224893/job/52592958289

Quarto version: 1.8.25
Render: Using current directory (/home/runner/work/issue-large-brand-yml/issue-large-brand-yml) as implicit input
projectContext: Found Quarto project in /home/runner/work/issue-large-brand-yml/issue-large-brand-yml
[execProcess] which git
[execProcess] Success: true, code: 0
[execProcess] git rev-parse
[execProcess] Success: true, code: 0

[1/4] index.qmd
Loaded deno-dom-native
[listing] Reading listing 'listing' from /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/index.qmd
[listing] Contents: posts
[listing] matches 2 files:
[listing] Reading file /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/posts/post-with-code/index.qmd
[listing] Reading file /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/posts/welcome/index.qmd
Creating SassCache at /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/sass
Returning SassCache at /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/sass
SassCache.getOrSet(...)
SassCache.getFromHash(hash=ed44c6ceb22dbcf0e2a7e62e499f535a, inputHash=b14ceb609f720c1e684d140a9175115c, force=undefined)
  cache miss
  cache miss, setting
SassCache.setFromHash(ed44c6ceb22dbcf0e2a7e62e499f535a, b14ceb609f720c1e684d140a9175115c), ...
[execProcess] /opt/quarto/bin/tools/x86_64/dart-sass/sass /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/d9613dbe81ada4ca.scss /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/sass/ed44c6ceb22dbcf0e2a7e62e499f535a.css --style compressed --quiet --load-path=/opt/quarto/share/formats/html/bootstrap/dist/scss --load-path=/opt/quarto/share/formats/html/bslib/bslib-scss
[execProcess] Success: true, code: 0
Returning SassCache at /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/sass
SassCache.getOrSet(...)
SassCache.getFromHash(hash=f3eea1ac85e5c51bb289a62823e8b8de, inputHash=2606fe75b784c9b08aba945778fdac54, force=undefined)
  cache miss
  cache miss, setting
SassCache.setFromHash(f3eea1ac85e5c51bb289a62823e8b8de, 2606fe75b784c9b08aba945778fdac54), ...
[execProcess] /opt/quarto/bin/tools/x86_64/dart-sass/sass /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/5851b6b2be9678d3.scss /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/sass/f3eea1ac85e5c51bb289a62823e8b8de.css --style compressed --quiet --load-path=/opt/quarto/share/formats/html/bootstrap/dist/scss --load-path=/opt/quarto/share/formats/html/bslib/bslib-scss
[execProcess] Success: true, code: 0
Returning SassCache at /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/sass
SassCache.getOrSet(...)
SassCache.getFromHash(hash=ed44c6ceb22dbcf0e2a7e62e499f535a, inputHash=b14ceb609f720c1e684d140a9175115c, force=undefined)
  cache hit
  cache hit
Creating SassCache at /home/runner/.cache/quarto/sass
Returning SassCache at /home/runner/.cache/quarto/sass
SassCache.getOrSet(...)
SassCache.getFromHash(hash=3944bc705cfaf4c80e56f77c5205168a, inputHash=2c2b845099e6708a022b169c06b17da8, force=undefined)
  cache miss
  cache miss, setting
SassCache.setFromHash(3944bc705cfaf4c80e56f77c5205168a, 2c2b845099e6708a022b169c06b17da8), ...
[execProcess] /opt/quarto/bin/tools/x86_64/dart-sass/sass /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/d3ff5a4f9575819a.scss /home/runner/.cache/quarto/sass/3944bc705cfaf4c80e56f77c5205168a.css --style expanded --quiet
[execProcess] Success: true, code: 0
Returning SassCache at /home/runner/.cache/quarto/sass
SassCache.getOrSet(...)
SassCache.getFromHash(hash=f5ab3bb6f5ded8da9f12dd3a91d3da26, inputHash=1d1d536d365c58781b22d669ce9e5151, force=undefined)
  cache miss
  cache miss, setting
SassCache.setFromHash(f5ab3bb6f5ded8da9f12dd3a91d3da26, 1d1d536d365c58781b22d669ce9e5151), ...
[execProcess] /opt/quarto/bin/tools/x86_64/dart-sass/sass /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/1d42efd67be999b2.scss /home/runner/.cache/quarto/sass/f5ab3bb6f5ded8da9f12dd3a91d3da26.css --style expanded --quiet
[execProcess] Success: true, code: 0
[NotebookContext]: Writing all notebooks to /home/runner/work/issue-large-brand-yml/issue-large-brand-yml/.quarto/quarto-session-temp8ca9d17d363d1d45/31a4284602c5fbad.json
-- Searching for R binary --
Looking for 'Rscript' in QUARTO_R: undefined
Looking for 'Rscript' in R_HOME: undefined
Looking for 'Rscript' in PATH.
[execProcess] which Rscript
[execProcess] Success: false, code: 1
Quarto did not find Rscript and will try to use it directly from the path.
[execProcess] /opt/quarto/bin/tools/x86_64/pandoc +RTS -K512m -RTS --defaults /tmp/quarto-session7a7e70754e86f52c/40233bb653eef122/quarto-defaultscee0dfd799451dcc.yml /tmp/quarto-session7a7e70754e86f52c/40233bb653eef122/quarto-inputda747758305f23e.md --metadata-file /tmp/quarto-session7a7e70754e86f52c/40233bb653eef122/quarto-metadata1a1716b5a96103e4.yml --verbose --trace --data-dir /opt/quarto/share/pandoc/datadir
[NotebookContext]: Starting Cleanup
ERROR: Error executing '/opt/quarto/bin/tools/x86_64/pandoc': Failed to spawn '/opt/quarto/bin/tools/x86_64/pandoc': Argument list too long (os error 7)

Stack trace:
    at execProcess (file:///opt/quarto/bin/quarto.js:5960:11)
    at file:///opt/quarto/bin/quarto.js:114112:18
    at runPandoc (file:///opt/quarto/bin/quarto.js:114114:25)
    at eventLoopTick (ext:core/01_core.js:178:7)
    at async renderPandoc (file:///opt/quarto/bin/quarto.js:115285:24)
    at async Object.onRender (file:///opt/quarto/bin/quarto.js:123976:30)
    at async renderFileInternal (file:///opt/quarto/bin/quarto.js:123953:9)
    at async renderFiles (file:///opt/quarto/bin/quarto.js:123684:9)
    at async renderProject (file:///opt/quarto/bin/quarto.js:124209:23)
    at async _Command.actionHandler (file:///opt/quarto/bin/quarto.js:129733:24)
    at async _Command.execute (file:///opt/quarto/bin/quarto.js:9840:7)
    at async _Command.parseCommand (file:///opt/quarto/bin/quarto.js:9717:14)
    at async quarto (file:///opt/quarto/bin/quarto.js:175526:5)
    at async file:///opt/quarto/bin/quarto.js:175554:5
    at async file:///opt/quarto/bin/quarto.js:175409:14
    at async mainRunner (file:///opt/quarto/bin/quarto.js:175411:5)
    at async file:///opt/quarto/bin/quarto.js:175547:3
Error: Process completed with exit code 1.

To reproduce, you can fork the repository and trigger the render workflow.

Project: https://github.com/mcanouil/issue-large-brand-yml

The project is:

• quarto create project blog
• large _brand.yml
• simple GHA workflow to render the project with debugging options enabled

[cderv]

It could be possible that QUARTO_FILTER_PARAMS environment variable is getting too big.

We use it to pass information to filters

quarto-cli/src/command/render/pandoc.ts

Lines 324 to 326 in c9c36f0

	pandocEnv["QUARTO_FILTER_PARAMS"] = encodeBase64(
	JSON.stringify(paramsJson),
	);

and the full brand object is among it

quarto-cli/src/command/render/filters.ts

Line 196 in c9c36f0

[kBrand]: options.format.render[kBrand],

If this is confirmed, maybe we could filter what is exposed to filters ?
We do this for some other configurations it seems. 🤔

quarto-cli/src/command/render/filters.ts

Lines 176 to 181 in c9c36f0

	...crossrefFilterParams(options, defaults),
	...citeIndexFilterParams(options, defaults),
	...layoutFilterParams(options.format, defaults),
	...languageFilterParams(options.format),
	...jatsFilterParams(options),
	...notebookContextFilterParams(options),

[cscheid]

It could be possible that QUARTO_FILTER_PARAMS environment variable is getting too big.

That's exactly what this bug is. We should be creating a temp file, writing that temp file to disk, and then pointing QUARTO_FILTER_PARAMS to the location of the file.