Is it possible to/How can I render a document-string from the commandline without creating the source doc file?

[Gewerd-Strauss]

Hello,

Assume I generate syntax-valid qmd-files programmatically, and then render those to output formats via the quarto commandline. That would simply be

quarto render doc.qmd --to html [opts]

I know that pandoc would allow me to render the file's string directly, that is without having the file itself, e.g.

printf "## Hello World\nThis is some markdown content." | pandoc -f markdown -t html -o output.html

I believe (but am not entirely certain right now) that all format-options which I would usually provide to quarto via a yml-file can be passed on the commandline as well.
This would mean that to my knowledge only the input-document file is strictly required to be a file, whereas everything else can be passed directly¹.

Could this be implemented though? I personally think this to be somewhat odd, but I am probably not aware of some reason or implication against this.

A potential short-time hack to implement night be to write the string to file, render it via quarto and then remove it again. However, I haven't tested this yet.

Would this be possible in principal, and would it be something you consider worth implementing, or does this fall outside the scope or design for quarto?

---

I also think that in theory this could allow rendering the same file string to different formats in parallel, but I am not sure to what degree file access and locking would interfere here.

---

Thank you.
Sincerely,
~Gw

---

Footnotes

1. Ignoring dependency files like images, data- and code-files, etc... for now. ↩

[cderv]

I believe (but am not entirely certain right now) that all format-options which I would usually provide to quarto via a yml-file can be passed on the commandline as well.
This would mean that to my knowledge only the input-document file is strictly required to be a file, whereas everything else can be passed directly1.

To be clear, not all configuration can be passed at command line. Some need to be passed through file, either .yml file, or in document.

Also, I supposed to you are focusing on single file render, and no project.

A potential short-time hack to implement night be to write the string to file, render it via quarto and then remove it again. However, I haven't tested this yet.

It seems a valid workflow to temporarily create what you need.

Could this be implemented though? I personally think this to be somewhat odd, but I am probably not aware of some reason or implication against this.

I won't answer on this as I don't know really. I'll let others in the team answer on that.

[Steinthal]

@Gewerd-Strauss, I agree with cderv's "However, I don't really see the use case to be honest. For document with a frontmatter, how would it be simpler to use command line only instead of using a file to store the string ?"

You could create your doc.qmdwith all needed definitions and {{< include _mydata.qmd >}}. Then simply create _mydata.qmdprogrammatically (overwriting it each time) and quarto render doc.qmd (assuming that the desired format is stated in doc.qmd).

If necessary, you could define metadata-file: mymeta.yml in doc.qmdand create mymeta.ymlprogrammatically as well.

Regarding "I also think that in theory this could allow rendering the same file string to different formats in parallel, but I am not sure to what degree file access and locking would interfere here.":

If you define several formats in doc.qmd (assuming that you need to define some specifics for each format), you could call quarto render doc.qmd --to <formatx> for each format defined one after the other.

I use a similar setup for some tasks.

[Gewerd-Strauss]

For document with a frontmatter, how would it be simpler to use command line only instead of using a file to store the string ?

The initial assumption was that all configuration can be provided and declared in the document's frontmatter. Since cderv noted that that was not the case, this whole thing wouldn't be worth it imo. In a perfect world I would like the option to render a document completely from the commandline, without requiring to provide any files at all.

---

Just curious about the full context of such feature you desire.

As background:

I write my documents in quarto-syntax using Obsidian.md, and use https://github.com/obsidian-html/obsidian-html to convert obsidian.md's syntax into standard markdown syntax (honestly, I just use it to combine transclusions), e.g.

## A header

![[another note]]
![[another note#section]]

This has a couple of advantages over quarto's {{< include _data.qmd >}}, namely that I can insert an arbitrary section of a note only; meaning I can write my manuscripts in per-method-notes à la

---
frontmatter
title: method_x
---

# Method X

# Results X 

# Discussion X

# Annex X

and insert the related things in my manuscript at the right locations.

There are more small things which are just faster for me in ObsidianMD than using VScode or Rstudio as an editor. If it's not code, Obsidian is just superior in my opinion, due to very mature plugin environment, templating, hotkeys for just about anything you can think of, and so on....

But to get from here to my rendered manuscript, I must go

1. obsidianMD_note (with mostly quarto-like syntax) > {Obsidian-HTML} > standard_markdown (preserved quarto-like syntax)
2. > {own utility for post-processing, final conversion steps I like to do myself} > qmd_file with complete qmd-valid syntax, + generates yaml-config files for each desired output-format, + a variety of other things required
3. > {quarto-cli} > rendered outputs

Overall, I generate 3-4 markdown-flavour files between feeding my original note to ObsidianHTML until I get my rendered outputs from quarto.

I am currently in the process of porting and refactoring my own utility from AHK to python, and am trying to cut down on this pipeline. It is somewhat messy, and became so because this was a growing project over the past 3-or-so-years. And while the ahk-instance is completely fine for my needs right now, modifying it becomes tiresome.

ObsidianHTML I can fork and modify myself to return the string of the converted file to stdout if I want, instead of writing it to file.
This means I can eliminate the need for files related to ObsidianHTML's functioning; and the only file-requirements are quarto's document (and yaml-configs) to get to the outputs.

So yea, it's a super specific usecase that makes me ask about this; but I believe that in general terms of automation it could be useful to not require the file itself. I would mod this myself, weren't it that quarto is waaaayy outside my expertise and I am not even remotely confident in just playing around in it.

---

Regarding "I also think that in theory this could allow rendering the same file string to different formats in parallel, but I am not sure to what degree file access and locking would interfere here.":

If you define several formats in doc.qmd (assuming that you need to define some specifics for each format), you could call quarto render doc.qmd --to <formatx> for each format defined one after the other.

Oh for sure. I do almost the same in my thesis projects, using ahk and R. This works fine, although it takes a bit. Particularly if you have a lot of documents. But doing so in parallel would probably fail due to file-locking if each quarto-process were to target the same source-doc. Just copying the document X times would allow this, but also be against the point of trying to reduce the number of required files.

[mcanouil]

Are you aware of the embed shortcode? https://quarto.org/docs/authoring/notebook-embed.html

And contents shortcode? https://prerelease.quarto.org/docs/authoring/contents.html

[Gewerd-Strauss]

Yes, but to my understanding

• {{< contents >}} only embeds code-cells/divs from a place in documentA into another place in documentA,
• and {{< embed >}} only embeds code-cells from documentA into documentB,

both of which are useful, but don't do what I want/need.

What I do is take an arbitrary chapter of documentA.md and insert it into the generated documentB.md at an arbitrary place. This allows me to

1. Structure documentA as a valid standalone document that I can render on my own, and still
2. take entire sections of it into documentB, anywhere I'd like. With a bit of trickery I can even include the body of a section while omitting the section-header itself.

This makes documentA renderable. I get a single document with the documentation on methods, results and discussion for a specfic method/experiment performed; and then in the overarching submission I can place the relevant sections in the relevant places.

This is a feature of ObsidianHTML.

Additionally, a transcluded section section of

![[note#section]]

can itself contain transclusions, with a declared recursion-level which gets resolved. I am not sure if the shortcodes listed by you can resolve text containing these shortcodes themselves ¯\_(ツ)_/¯

And finally, doing it via obsidianhtml means I can preview the result via obsidian.md's preview/live-edit modes, as those render these transclusions. It helps sometimes.

Maybe I could set up a similar workflow somehow using quarto alone, but using ObsidianMD is just too convenient for me. I write almost no code in-document for my manuscripts, instead sourcing scripts extensively and only using codeblocks for formatting, figures, tables, and inline-code for, well inline-rendering. This means my writing- and coding-processes are mostly decoupled to different files; instead of doing it all in the document itself. For these reasons, I rarely have a usecase for transcluding codeblocks alone.

[Steinthal]

{{< contents >}} only embeds code-cells/divs from a place in documentA into another place in documentA,

Well, that's not exactly true. You can do the following:

_documentA-inc.qmd:

::: {#method-x}
# Method X
Content of method-x

More content…
:::

::: {.content-visible when-meta="documentA"}
# Results X 
:::

::: {#results-x}
Content of results-x
:::


::: {#discussion-x}
# Discussion X
Content of discussion-x

More content…
:::


::: {#annex-x}
# Annex X
Content of annex-x

More content…
:::

documentA.qmd -- to define the title for the "standalone" document:

---
title: "Doc X"
format: html
documentA: true
---

{{< include _documentA-inc.qmd >}}

documentB.qmd -- the document combining several sections:

---
title: "Main"
format: html
---

{{< include _documentA-inc.qmd >}}

# Method M
{{< lipsum 1 >}}

{{< contents method-x >}}


# Results M 
{{< lipsum 1 >}}

## Results taken from document X
{{< contents results-x >}}


# Discussion M
{{< lipsum 1 >}}

{{< contents discussion-x >}}


# Annex M
{{< lipsum 1 >}}

{{< contents annex-x >}}

Result

DocumentA

DocumentB

Probably still not what you need and maybe too complex, but for me {{< contents "dat" >}} offers some interesting possibilities.