How to use R and Python in the same quarto doc?

[durraniu]

Description

I am trying to use R and Python in the same quarto doc. But reticulate throws an error and it seems that it does not remember the data for Python in the previous code chunk. I have run the Python and R code separately and it runs. If I change the engine to jupyter, all the Python code chunks run successfully (but R code chunks don't run). My quarto version is 1.3.450.

Here is my quarto doc:

---
title: "Transform Data"
date: "2024-01-20"
categories: [import, transform, dplyr, pandas, polars, arrow, janitor]
image: "trajectories.gif"
code-annotations: hover
engine: knitr
---

```{r setup}
#| echo: false
reticulate::use_python("C:\\Users\\umair\\AppData\\Local\\Programs\\Python\\Python312\\python.exe")
```


```{r}
#| eval: false

library(arrow)
library(dplyr)

path_to_partitioned_first_period_file <- "D:/ngsim/posts/import/data/I80/period=first/part-0.parquet"

open_dataset(path_to_partitioned_first_period_file) |>
  dplyr::collect() |>
  arrow::write_parquet(sink = "data/i80_period1.parquet")
```


## Load parquet file 

::: {.panel-tabset group="language"}
### R
```{r rread}
library(arrow)
library(dplyr)

path_to_first_period_file <- "data/i80_period1.parquet"
data_R <- arrow::read_parquet(file = path_to_first_period_file)

data_R
```

### Python
```{python pyread}
import polars as pl

path_to_first_period_file = "data/i80_period1.parquet"
data_py = pl.read_parquet(path_to_first_period_file)

data_py.head()
```
:::


## Clean dataframe names

::: {.panel-tabset group="language"}
### R
```{r rclean}
library(janitor)
data_R <- janitor::clean_names(data_R) 
```

### Python
```{python pyclean}
import pandas as pd
import pyarrow
import janitor
data_py = data_py.to_pandas()
data_py = data_py.clean_names()

data_py.head()
```
:::

Result of quarto background job:

==> quarto preview transform.qmd --to html --no-watch-inputs --no-browse

[1/2] posts\transform\transform.qmd


processing file: transform.qmd
  |.................................                   |  63% [pyclean]        
Quitting from lines 67-74 [pyclean] (transform.qmd)
Error in `py_call_impl()`:
! ImportError: DLL load failed while importing lib: The specified procedure could not be found.
Run `reticulate::py_last_error()` for details.
Backtrace:
  1. global .main()
  2. execute(...)
  3. rmarkdown::render(...)
  4. knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
  5. knitr:::process_file(text, output)
     ...
 15. knitr (local) engine(options)
 16. reticulate::eng_python(options)
 17. reticulate:::py_compile_eval(snippet, compile_mode)
 20. builtins$eval(compiled, globals, locals)
 21. reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
                                                                                                            
Execution halted

[cderv]

I can't try you example because you are using data that is not available to us.

So let me just comment on your question with some details hoping it helps. Please to provide reproducible example if you want further help.

But reticulate throws an error and it seems that it does not remember the data for Python in the previous code chunk.

This is the error I see in your OP:

! ImportError: DLL load failed while importing lib: The specified procedure could not be found.

This is related to something incorrectly loading when running the code with reticulate in a knitting context.

You should try to isolate the code that fails in this last pyclean chunk to run with reticulate.

I think the issue is related to reticulate R package. You can try in R Markdown or knitr directly to confirm Quarto is not involved.

• Running python for R console: https://rstudio.github.io/reticulate/articles/calling_python.html
• Running python for python cells with knitr and markdown: https://rstudio.github.io/reticulate/articles/r_markdown.html

This could help debug.

I have run the Python and R code separately and it runs.

All cells runs individually in python or with reticulate ? Do they run interactively in RStudio IDE ?

If I change the engine to jupyter, all the Python code chunks run successfully (but R code chunks don't run).

This is expected. With engine Jupyter, a single kernel can be run, and so R cells are not executable. knitr engine is specific for that because of support of reticulate for running Python cells.

My quarto version is 1.3.450.

Not sure it is related to Quarto, but still useful to update to latest pre-release version 1.4 currently Release Candidate: https://quarto.org/docs/download/

Hope it helps

[durraniu]

Thanks for your answer. It was not a quarto issue, but a reticulate one.