Is it possible to use multiple engines if e.g. R and python are isolated in separate includes?

[jwhendy]

Description

Related: #4232, #8253

From my reading, it seems to use multiple languages in the same project, one needs to use an engine capable of both, e.g. the knitr engine with reticulate. I've had mixed results with reticulate and would prefer to use straight python, if possible.

If what I show below is simply not possible, consider me testing the waters of a feature request. We are structuring a living analysis document, and I envision members contributing by taking a narrow hypothesis and exploring it various ways. Each will be an isolated, single-language "card", with all analysis cards included via {{ < include _foo.qmd > }} in a main document as we write them.

Is there no setup/set of options where I could do something like this?

main_document.qmd

---
title: "Test"
execute:
  eval: false
---

# Main document
{{< include section_R.qmd >}}
{{< include section_py.qmd >}}

section_R.qmd

---
execute:
  cache: true
  freeze: true
---

# Section using R code
```{r}
print("this is R code")
```

section_py.qmd

---
execute:
  freeze: true
  cache: true
jupyter: python3
---

# Section using python code
```{python}
print('this is python code')
```

My thinking was that if one could cache/freeze the results of the code-containing .qmd cards, the top level document would not need to invoke an engine at all, but simply assemble the parent/children htmls.

Is there any appetite for something like this? It would be a dream if quarto had an option to load up the R engine, compile all includes which set the engine to knitr and do the same for all that set the engine to jupyter/python.

Looking at the questions from @cscheid in #4232 (the closest existing discussion I found), here's a stab:

how do we resolve the order in which the kernels are selected? What happens when they overlap in the cells that execute?

I am proposing a single-language-per-file approach, so order doesn't matter (in as much as order does/doesn't matter in any other single-language use case). Maybe this is not sufficiently compelling, as I just want to avoid reticulate.

how do we actually schedule the execution? Repeated calls into nbclient would require a substantial reworking of our execution codepath.

Functionally, this would be like having two files or two projects with only R or python includes, rendering both separately, then stitching together sections from the resultant htmls in one's desired order.

how do we teach users about this? We already have a large source of confusion in the case where both python and r code cells are present in the knitr engine (which uses the R reticulate package).

I don't have enough context for the confusion you've already faced, but trivially: add more to your [already comprehensive/pretty excellent] docs :)

will this actually work like users expect? If users expect cells to communicate with each other, then this will need support from the kernels (which by and large won't expect to be executing alongside other kernels)

I'm cheating by rendering this a non-question. In my case, the path I was headed down was to have separate R and python files load from our db, do some ETL, and then make objects of the same name available in either environment. In this way users can write up their analysis "card" and assume that e.g. df includes with the same structure, whether they want to do the analysis in python or R.

That's done in some setup code, which from my testing seems to be available to all subsequent includes [using the same engine].

Thanks for taking a look; again, maybe avoiding reticulate is a pretty silly motivation. I've run into some oddities before in brief exploration of it and it just feels cleaner to use python directly. Maybe others have alternative use cases for this, however, as there's a decent number of questions about mixed languages (e.g. scenarios where a single engine doesn't exist which can handle both languages).

[mcanouil]

As documented, the include shortcode is a copy/paste shortcode, nothing more.

If you don't like knitr/reticulate, you can do Jupyter/rpy2 (https://rpy2.github.io/doc/latest/html/introduction.html).

What you want seems to be the embed shortcode: https://quarto.org/docs/authoring/notebook-embed.html

[jwhendy]

As documented, the include shortcode is a copy/paste shortcode, nothing more.

Fair, though I proposed a second lens :)

If what I show below is simply not possible, consider me testing the waters of a feature request.

I can check out the other mix/match. I haven't used rpy2 before.

What you want seems to be the embed shortcode:

From a look, it seems this is at the code level vs. at the markdown section level. Is that accurate? This might work, and I wasn't aware of this. I could envision writing all the prose in the main document, then keep all the code-generated results in single-language documents, then just bring in the e.g. tables/figures where they belong. Thanks for the pointer.