How to link a specific quarto website to a specific virtual environment of python?

[emmansh]

Building a quarto website, I want my index.qmd file to include python code chunks. Specifically, I want to link this .qmd file to a virtual environment of python. Furthermore, I wish to use python's venv thus avoiding reticulate or conda.

The problem

Doing all the steps as I will describe below, I end up with quarto not recognizing python's venv, despite being in the same project folder. That is, running in Terminal the command:

# Terminal
quarto check jupyter

I get

# Terminal output
## [>] Checking Python 3 installation....(None)G
##       Unable to locate an installed version of Python 3.
##       Install Python 3 from https://www.python.org/downloads/

Steps to reproducing the problem

I have a clean venv setup that I adopted from this YouTube video.

I'm installing this on a Windows machine

Install python 3

1. Download python from https://www.python.org/downloads/
2. Using the installation wizard, I specify a custom path for this current instance of python (in my case, v 3.9.0; 64-bit).
   • so the destination folder for this python installation is: C:\pyver\py390
   • I also ensure to not add this python installation to PATH. I want everything to be as "isolated" as possible.

Create quarto website

1. RStudio -> File -> New Project -> New Directory -> Quarto Website
   

2. I give it a name: my-qrt-website
   

3. Now I have a minimal setup for quarto website, and I want to add python code chunks. At this point, quarto can't detect any python installation:

   # Terminal
   quarto check

   I get

   # Terminal output
   ## [>] Checking Quarto installation......OK
   ##       Version: 1.2.258
   ##       Path: C:\Users\emman\AppData\Local\Programs\Quarto\bin
   ##       CodePage: 1252
   
   ## [>] Checking basic markdown render....OK2K←[G
   
   ## [>] Checking Python 3 installation....(None)
   
   ##       Unable to locate an installed version of Python 3.
   ##       Install Python 3 from https://www.python.org/downloads/
   
   ## [>] Checking R installation...........OK2K←[G
   ##       Version: 4.2.1
   ##       Path: C:/PROGRA~1/R/R-42~1.1
   ##       LibPaths:
   ##         - C:/Program Files/R/my_packages
   ##         - C:/Program Files/R/R-4.2.1/library
   ##       rmarkdown: 2.17
   
   ## [>] Checking Knitr engine render......OK2K←[G

I assume, but didn't confirm it, that quarto is unable to locate the python I installed earlier because I didn't add it to PATH. As far as I'm concerned, this is very good, because I do not want quarto to pick up whatever python installation it finds on my machine. Rather, I want this specific quarto website to link with a specific python version and a specific virtual environemnt (i.e., python modules of specific versions, etc.).

This is why I found quarto's documentation about virtual environments very promising. Specifically, the piece that says:

RStudio will automatically activate any venv or condaenv that it finds within a project directory. Just be sure to create an RStudio project within the same directory where you created your env and things will work as expected with no additional configuration [ref].

Setting up virtual environment with venv

1. In Terminal, navigate to quarto website directory

   • so command-line prompt says D:\local_quarto_website\my-qrt-website>

2. Create a virtual environment called env, while referring to the desired, specific, python version:

   # Terminal    
   c:\pyver\py390\python -m venv env

3. After a couple of seconds, we can see that a new folder called env has been created:
   

4. Activate the virtual environemnt

   # Terminal
   env\Scripts\activate

   And command-line prompt changes to (env) D:\local_quarto_website\my-qrt-website>

5. Within the now-activated virtual environment, install python modules

   # Terminal
   pip install jupyter matplotlib pandas

6. Deactivate the virtual environment

   # Terminal
   deactivate

Circle back to the problem

So now I have a quarto website directory with the following contents:

 # Terminal
 D:\local_quarto_website\my-qrt-website>dir
 ## Volume in drive D is Data
 ## Volume Serial Number is 2475-30F4
 ## 
 ## Directory of D:\local_quarto_website\my-qrt-website
 ## 
 ## 11/06/2022  11:08 AM    <DIR>          .
 ## 11/06/2022  11:08 AM    <DIR>          ..
 ## 11/06/2022  10:30 AM                58 about.qmd
 ## 11/06/2022  11:16 AM    <DIR>          env
 ## 11/06/2022  10:30 AM               156 index.qmd
 ## 11/06/2022  10:30 AM               218 my-qrt-website.Rproj
 ## 11/02/2022  01:34 PM                17 styles.css
 ## 11/06/2022  10:30 AM               227 _quarto.yml
 ##                5 File(s)            676 bytes
 ##                3 Dir(s)  93,502,058,496 bytes free

So why can't it recognize my python virtual environment, despite having this nice env directory as promised in the documnetation?

# Terminal
D:\local_quarto_website\my-qrt-website>quarto check jupyter

## [>] Checking Python 3 installation....(None)

##       Unable to locate an installed version of Python 3.
##       Install Python 3 from https://www.python.org/downloads/

[emmansh]

I found this bug report that gave me a clue to follow. Although I made some progress with it, I'm still unable to solve the problem.

Setting up the QUARTO_PYTHON environment variable

Since I didn't add the python installation to PATH, it's plausible that quarto check returns that no python was found. Therefore, I went ahead and set QUARTO_PYTHON environment variable to point directly at C:\pyver\py390\python.exe.

# Windows PowerShell
$env:QUARTO_PYTHON=$("C:\pyver\py390\python.exe")

Now, quarto check finds my python installation:

# Windows PowerShell
quarto check

## [>] Checking Quarto installation......OK
##       Version: 1.2.258
##       Path: C:\Users\emman\AppData\Local\Programs\Quarto\bin
##       CodePage: 1252
## 
## [>] Checking basic markdown render....OK2K←[G
## 
## [>] Checking Python 3 installation....OK
##       Version: 3.9.0
##       Path: C:/pyver/py390/python.exe
##       Jupyter: (None)
## 
##       Jupyter is not available in this Python installation.
##       Install with python -m pip install jupyter
## 
##       There is an unactivated Python environment in env. Did you forget to activate it?
## 
## [>] Checking R installation...........OK2K←[G
##       Version: 4.2.1
##       Path: C:/PROGRA~1/R/R-42~1.1
##       LibPaths:
##         - C:/Program Files/R/my_packages
##         - C:/Program Files/R/R-4.2.1/library
##       rmarkdown: 2.17
## 
## [>] Checking Knitr engine render......OK2K←[G

So it found python, but couldn't find Jupyter. That makes sense, because I didn't install jupyter globally on my machine. Rather, I installed jupyter within my virtual environment. Indeed, it seems that my python modules, and jupyter among them, are buried inside the virtual environment directory (i.e., env) and are inaccessible to quarto.

There is an unactivated Python environment in env. Did you forget to activate it?

Do I need to activate the environment? I thought that it should be detected automatically by RStudio/VSCode, as long as it's called env and it's in the project directory.

But actually, even if I do activate the virtual environment, quarto still doesn't find jupyter:

# Windows PowerShell
# code from: https://stackoverflow.com/a/59815372/6105259
Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope Process -Force
./env/Scripts/activate.ps1

Now PowerShell prompt has changed to

(env) PS D:\local_quarto_website\my-qrt-website>

So we know that the virtual environment is activated. So we are going to execute quarto check from within the virtual environement, but to no avail. It still won't recognize jupyter:

(env) PS D:\local_quarto_website\my-qrt-website> quarto check

## [>] Checking Quarto installation......OK
##       Version: 1.2.258
##       Path: C:\Users\emman\AppData\Local\Programs\Quarto\bin
##       CodePage: 1252
## 
## [>] Checking basic markdown render....OK2K←[G
## 
## [>] Checking Python 3 installation....OK
##       Version: 3.9.0
##       Path: C:/pyver/py390/python.exe
##       Jupyter: (None)
## 
##       Jupyter is not available in this Python installation.
##       Install with python -m pip install jupyter
## 
##       There is an unactivated Python environment in env. Did you forget to activate it?
## 
## [>] Checking R installation...........OK2K←[G
##       Version: 4.2.1
##       Path: C:/PROGRA~1/R/R-42~1.1
##       LibPaths:
##         - C:/Program Files/R/my_packages
##         - C:/Program Files/R/R-4.2.1/library
##       rmarkdown: 2.17
## 
## [>] Checking Knitr engine render......OK2K←[G

Is this a bug? Why does the virtual environment remain obscure to quarto? I have tested this procedure with both RStudio & VSCode, and the behavior is the same in both.

[emmansh]

I made a little bit of progress that might help in troubleshooting. However, the main problem remains unsolved.

VSCode Quarto Extension's Render button

Suprisingly, if I open my index.qmd in VSCode and hit Render (provided by the VSC Quarto extension), it causes quarto to recognize my virtual environemnt. Upon clicking Render, a new, distinct Terminal window opens up. In VSCode, this Terminal window is titled Quarto Preview, despite being a Windows PowerShell process, essentially.

Although this rendering process fails (I'll elaborate on that below), it nevertheless — somehow — finds the virtual environment such that:

# Windows PowerShell / "Quarto Preview"
quarto check

returns

# Windows PowerShell / "Quarto Preview"

## [>] Checking Quarto installation......OK
##       Version: 1.2.258
##       Path: C:\Users\emman\AppData\Local\Programs\Quarto\bin
##       CodePage: 1252
## 
## [>] Checking basic markdown render....OK2K←[G
## 
## [>] Checking Python 3 installation....OK2K←[G
##       Version: 3.9.0
##       Path: d:/local_quarto_website/my-qrt-website/env/Scripts/python.exe
##       Jupyter: 4.11.2
##       Kernels: python3
## 
## [>] Checking Jupyter engine render....OK2K←[G
## 
## [>] Checking R installation...........OK2K←[G
##       Version: 4.2.1
##       Path: C:/PROGRA~1/R/R-42~1.1
##       LibPaths:
##         - C:/Program Files/R/my_packages
##         - C:/Program Files/R/R-4.2.1/library
##       rmarkdown: 2.17
## 
## [>] Checking Knitr engine render......OK2K←[G

By contrast, the Render button in RStudio IDE does not help in recognizing the virtual environment.

VSCode Render button doesn't fulfill its goal

Although it does help, magically, to recognize the virtual environment, Render button doesn't lead to a rendered webpage. It throws this full Terminal output:

# Windows PowerShell / "Quarto Preview"

## Windows PowerShell
## Copyright (C) 2014 Microsoft Corporation. All rights reserved.
## 
## PS D:\local_quarto_website\my-qrt-website> & d:/local_quarto_website/my-qrt-website/env/Scripts/Activate.ps1
## & : File D:\local_quarto_website\my-qrt-website\env\Scripts\Activate.ps1 cannot be loaded because running scripts is disabled on this 
## system. For more information, see about_Execution_Policies at http://go.microsoft.com/fwlink/?LinkID=135170.
## At line:1 char:3
## + & d:/local_quarto_website/my-qrt-website/env/Scripts/Activate.ps1
## +   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
##     + CategoryInfo          : SecurityError: (:) [], PSSecurityException
##     + FullyQualifiedErrorId : UnauthorizedAccess
## PS D:\local_quarto_website\my-qrt-website> quarto preview d:/local_quarto_website/my-qrt-website/index.qmd --no-browser --no-watch-inputs
## 
## Starting python3 kernel...Done
## 
## Executing 'index.ipynb'
##   Cell 1/1...Done
## 
## pandoc
##   to: html
##   output-file: index.html
##   standalone: true
##   title-prefix: my-qrt-website
##   section-divs: true
##   html-math-method: mathjax
##   wrap: none
##   default-image-extension: png
##   css:
##     - styles.css
##   toc: true
## 
## metadata
##   document-css: false
##   link-citations: true
##   date-format: long
##   lang: en
##   editor: visual
##   theme: cosmo
##   title: my-qrt-website
## 
## Output created: _site\index.html
## 
## Terminating existing preview server....DONE
## 
## Preparing to preview
## 
## Watching files for changes
## Browse at http://localhost:4975/

The end looks promising, but when I browse to http://localhost:4975/ I get an error (in both my Chrome browser and in the terminal session):

ERROR: PermissionDenied: Access is denied. (os error 5), stat 'D:\local_quarto_website\my-qrt-website\index_files'