problem finding Python libraries in polygot (Python/R) document

[bbolker]

Bug description

No response

Checklist

• Please include a minimal, fully reproducible example in a single .qmd file? Please provide the whole file rather than the snippet you believe is causing the issue.
• Please format your issue so it is easier for us to read the bug report.
• Please document the RStudio IDE version you're running (if applicable), by providing the value displayed in the "About RStudio" main menu dialog?
• Please document the operating system you're running. If on Linux, please provide the specific distribution.

[bbolker]

I'm running into trouble locating Python libraries in a polyglot document, i.e. one that contains both R and Python code chunks. This fails with Quarto 1.1.335 but works on another machine with Quarto 1.1.269 (both running Pop!OS 22.04). (No RStudio environment, running quarto from the command line)

Here's my .qmd file:

---
title: "polyglot"
---

```{python split}
import pandas as pd
```

```{r test}
print("hello world")
```

If I quarto render this I get

Quitting from lines 8-9 (test.qmd)
Error: ModuleNotFoundError: No module named 'pandas'
Execution halted

If I delete the R chunk rendering works: I get

Starting python3 kernel...Done ... Executing 'test.ipynb' ... Cell 1/1...Done

---

quarto check output:

[✓] Checking Quarto installation......OK
Version: 1.2.335
Path: /opt/quarto/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
Version: 3.10.6
Path: /usr/bin/python3
Jupyter: 5.2.0
Kernels: python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
Version: 4.3.0
Path: /usr/local/lib/R
LibPaths:
- /usr/local/lib/R/site-library
- /usr/local/lib/R/library
rmarkdown: 2.20

[✓] Checking Knitr engine render......OK

---

The other install (on which rendering works) is Jupyter 4.9.1 but otherwise the same (at least as far as quarto check is concerned)

[cderv]

When using R with python, the package reticulate (https://rstudio.github.io/reticulate/) will be used. Considering the issue you have (Error: ModuleNotFoundError: No module named 'pandas'), it seems highly possible that Jupyter and R through reticulate are not using the same python environment to run your code.

Have you configure reticulate ?

You can either configure the reticulated python environment to install pandas, or you can configure the project so that the Python environment used by Jupyter and R with reticulate are the same. You can find more information on Python versions used with R in https://rstudio.github.io/reticulate/articles/versions.html . Doc about Python packages helps understand how the environment is used (https://rstudio.github.io/reticulate/articles/python_packages.html). I think reticulate will default to using a r-reticulate environment for Packages and that is probably different from what your Jupyter is using.

Using a virtual environment or any other method described in doc (like setting RETICULATE_PYTHON to the python version you want to use) should solve the issue.

[cscheid]

The other install (on which rendering works) is Jupyter 4.9.1 but otherwise the same (at least as far as quarto check is concerned)

Note that quarto check only checks if the very simplest executions will work. Specifically, quarto check will not verify which packages are installed in the Python environment it finds, because it cannot know ahead of time which packages will be required by which .qmd files.

In your case, the compilation is failing in import pandas, like Christophe mentioned. Since you're seeing different jupyter versions in the different quarto check versions reported, it is likely that one of the python setups will have pandas installed, while the other won't.

[bbolker]

Thanks, I think "Python + R uses reticulate" is probably the clue I needed as to why it works differently depending on whether an R chunk is included anywhere in the document - will check when I get back to the machine that's failing; your links look like they will be useful.

[cscheid]

(Moving this to Q&A since this is a python-setup question.)

[bbolker]

I can believe this is a python-setup question, but I don't quite understand (1) why it works in a .qmd file that is pure-Python and fails when I add a single R chunk and (2) why it works with Quarto 1.1.269 and fails with Quarto 1.1.335. Admittedly Python setup is very tricky and I can't rule out subtle differences in the Python environments between these two machines, beyond the Quarto-version differences. I guess I could start working in Docker containers to try to isolate the difference but I'm not sure I have the time to devote to that right now ...