.md files with executable code

[rmhorne]

Hello everyone,

First, this a great tool! At the moment I am trying to work on a workflow/ pipeline using Quarto that does the following:

1. Allows note taking in Obsidian with Zotero integration and code snippets that create some output (graphs, charts, maps, etc).
2. Publish the results in Quarto.

Essentially I want to be able to use the normal Obsidian note taking process (with the on-the-fly creation of new notes, the network view, etc), add in code blocks (in my case python) that can be rendered in quarto, and then publish the results. I am ~90% of the way there, but I have run into some issues, and maybe this list (or the devs!) could help.

My biggest issue / roadblock is Quarto's limitation on running executable code in .md files. Although I can just change the extension on all of my .md files to .qmd when I want to render them (and the results are great!), doing so breaks my linkages within my documents and some of the functionality within Obsidian. It would be a lot more convenient if I could just use something like

engine: jupyter

in the project .yaml file (or a yaml block within the file itself) that allows Quarto to render code in a .md file. Is there any way to override the current block on .md code execution, or any future plans to allow it? I realize that this could also be solved in the other direction by more extensive support for .qmd files in Obsidian, but I think this kind of functionality might be useful for other workflows / processes that have code in .md files.

Thanks!

[cscheid]

Currently, we assume .md files are intermediate markdown files pretty extensively throughout the codebase. I wouldn't want to rely on using .md files as input because there's a risk some part of our codebase will step on those files. (This is something we will 100% fix in the future, but it'll be a while until we can get to the bottom of it.)

I would recommend you save those with some other extension, possibly through a project script that happens automatically ahead of quarto render.

[rmhorne]

Thanks, and I'm glad to hear that it is something that will be fixed in the future!

A project script could work - in general I was trying to construct this pipeline for an academic humanities / non technical audience, but a quick script to change extensions, then crawl through .md files and change the relative links there before rendering might be a good way forward.

[cscheid]

You can create custom project types in extensions, so your audience should need very little to no configuration to get things started: https://quarto.org/docs/extensions/project-types.html

[jeffkimbrel]

I've been dabbling with trying to better integrate executable documents in R with stand alone markdown in Obsidian, and too have been struggling. I used to have a better solution with R markdown, but haven't found a solution with Quarto. What worked for me in Rmarkdown was to have the .rmd file outside of Obsidian, but then render to a markdown file inside of my Obsidian folder structure. Knitting allowed me to send the file somewhere via the front matter...

knit: (function(inputFile, encoding) {
    rmarkdown::render(inputFile, 
      encoding = encoding, 
      output_dir = "/PATH/TO/OUTPUT/)
    })

Figures could be a little wonky, as the links to them weren't always preserved exactly, but I could live with it. I would also have a callout in the output file saying this was just a rendering, and where to find the "real" document should I need to edit and run again.

For Quarto, I've been having trouble getting a markdown file to be written as "complete" as it was previously, and I haven't been able to send it to a different folder. Any suggestions? I'm also open to other ways to get Quarto and Obsidian to integrate.

[mcanouil]

Not clear to me how what you are trying to do differs from the initial message here.

An answer has been provided by Quarto team stating that you can't do what you want, i.e., rendering md files, etc.

Regarding output dir, the R package is for now not up to date.
However, you could call Quarto CLI yourself using for example system().
quarto render file.qmd --output-dir PATH/TO/OUTPUT

[jeffkimbrel]

I was just trying to show I had solved the original question problem (as I understand it) by flipping the source and destination around. I think the original question was rendering from .md inside Obsidian to an external file, where I was suggesting instead rendering from .qmd outside of Obsidian to .md inside Obsidian. I'm not trying to execute .md, rather write to .md.

Thanks for the idea of just command line rendering... I live and die by the "render" button, but I don't need to be!

A last idea for the original message... There is an obsidian plug in (txt-as-md) that can be modified to show non-.md files in Obsidian... so you can just have your .qmd file itself live in your obsidian vault and see it as though it is markdown. I've asked a search plug-in to find them as well: scambier/obsidian-omnisearch#58

I also found this, but doesn't seem tested: https://github.com/odebroqueville/obsidian-qmd-as-md

[mcanouil]

Thanks for the detailed explanation.

If you want to output markdown, you can use gfm format (or any markdown based format from Quarto/Pandoc) such as the one described in #4436.

[jeffkimbrel]

Thanks, some good links to follow on there, particularly setting a destination for images.

[baptiste]

For the quarto publishing step, I think you could simply include the .md file in a dummy .qmd wrapper, as in

---
title: "including"
format: html
---

{{< include _includee.md >}}

where _includee.md has the content,

Quarto enables you to weave together content and executable code into a finished document. To learn more about Quarto see <https://quarto.org>.

## Running Code

When you click the **Render** button a document will be generated that includes both content and the output of embedded code. You can embed code like this:

```{r}
1 + 1
```

You can add options to executable code like this 

```{r}
#| echo: false
2 * 2
```

The `echo: false` option disables the printing of code (only output is displayed).

[mcanouil]

This does not solve the possible between files "linkage" issue described in the first post.

[baptiste]

it's unclear to me what linkages are discussed in that first post: I took it as meaning the links created internally by Obsidian, in which case I assume they should still work (but I'd need to see a concrete example to know how Quarto would interpret them – if they're just relative paths, then it should be fine, I think).

The other way around – quarto-generated output, etc. – well it's not clear from the OP that Obsidian would ever see them in this workflow, since that would require modifying in-place the original input md document (which cannot be run again, so a destructive operation). If a new document is created and imported in Obsidian for the rendered output, then it should be enough to specify the right flavour of markdown as quarto/pandoc output format.

[mcanouil]

Exactly, as I pointed out a moment ago #3715 (reply in thread)

PS: I am not at all familiar with Obsidian.

[mcanouil]

FYI (@cderv originally posted in #5449)

Obsidian related discussion https://forum.obsidian.md/t/quarto-support-open-source-scientific-and-technical-publishing-system/32625/36 and Obsidian plugin https://github.com/danieltomasz/qmd-as-md-obsidian

Just for reference for anyone looking for Obsidian and Quarto

[nichtich]

My failed workaround to this problem was to create symlinks from each file.qmd to file.md and explicitly exclude the .md files in config:

project:
  render:
    - "!*.md"

But this fails with an AlreadyExists error. Any suggestions how to allow keeping a .md file as source is welcome!

[cderv]

What are you trying to do exactly ? Maybe you should open a new discussion about your use case.

.qmd is the source file for Quarto, it is like .md but with special Markdown content that Quarto can handle, (like computational code block). It will create a .md from this .qmd containing all the results from your evaluation, so that it can be converted using Pandoc to an output format.

This is why you may have issue about already exist error if you have .md with same name as .qmd.

If you want to keep the intermediate .md produced by a .qmd rendering you can use keep-md: true - though it will be using some Markdown syntax specific to what Quarto needs for its feature.

Hope it helps

[roaldarbol]

I think I'm running into a similar problem. Big project with many individual sections and includes, pretty much everything is common markdown. Then I want a single mermaid graph and I run into an error. I produce it in a separate mermaid-graph.qmd file which I am then attempting to import (in order not to have to convert to everything toqmd. But it seems that for it to work, every single file above it needs to also be qmd files - is that right?

I've made a minimal example in https://github.com/roaldarbol/qmd-in-md; rendering from main.qmd works, whereas rendering from main.md fails.

[roaldarbol]

I can of course also just render the diagrams separately and include them as figures - just seemed it might have been possible. 😊

[cderv]

I assumed it would render and the content be included. Does it simply include the raw content and then only render at the end?

Yes the include shortcode acts as a copy paste. It will include the content as raw and then only render at the end.

[roaldarbol]

Good to know, thanks! I'll just render them separately then, no problemo.

And actually, I guess that also means the really I just need the top-level document to be a qmd document (as opposed to all the intermediate documents); just tested, and that's indeed the case. :-)

[mcanouil]

You could try the embed shortcode instead of the include one: https://quarto.org/docs/authoring/notebook-embed.html

[roaldarbol]

I like that approach! It seems to be a bit finicky though, at least for my use case; I'm attempting to make a figure with 3 mermaid diagrams as separate subfigures. Embed seems to work for figure divs, BUT only with a single code block inside - adding more breaks the rendering. I've updated the reprex: https://github.com/roaldarbol/qmd-in-md

(As an aside, embedding starts a new section number for me - I don't know whether that's reported)