How do I create a custom computation/ filter that supports same execution options as {python} filter

[MarcSkovMadsen]

Description

Hi. I'm trying to make it super simple to use the HoloViz ecosystem with Quarto. See https://github.com/awesome-panel/holoviz-quarto.

Instead of using the {python} filter I would like to provide some custom filters. I want them to work in the same way as the {python} filter, i.e. taking the same execution options. I would like to be able to use echo, code-fold etc. options.

For example something like

```{panel-convert-python}
#| echo:true
import panel as pn

pn.extension(design="material")

slider = pn.widgets.IntSlider(name="Select a value", value=10, start=0, end=100)
pn.Column(
    pn.rx("You selected: {}").format(slider),
).servable()
```

Currently I can only get the rendered output

I will use Python to create the raw html output

local file_path = debug.getinfo(1, "S").source:sub(2)
local file_dir = file_path:match("(.*[/\\])")
local script_to_iframe_path = file_dir .. "script_to_iframe.py"

function CodeBlock(el)
  if el.attr and el.attr.classes:find_if(function (c) return string.match(c, "{?panel%-convert%-python}?") end) then
      converted_code = pandoc.pipe("python", {script_to_iframe_path}, el.text)
      quarto.doc.add_html_dependency(
        {
          name = "quarto-panel-convert",
          stylesheets = {"quarto-panel-convert.css"},
          scripts = {"quarto-panel-convert.js"},
        }
      )

      return pandoc.RawBlock("html", converted_code)
  end
end

I've tried to figure out how I can most easily do this, inheriting as much existing functionality as possible. But I've not been able to find this in the official docs or looking on github. I need help.

How do I create something like this?

[mcanouil]

I am not sure to follow why you want to define a new Jupyter engine kernel (see online resources about that since it's independent of Quarto).

If you want to add a class to easily target, use classes then use your filter.

```{python}
#| echo: true
#| classes: panel-convert-python
import panel as pn

pn.extension(design="material")

slider = pn.widgets.IntSlider(name="Select a value", value=10, start=0, end=100)
pn.Column(
    pn.rx("You selected: {}").format(slider),
).servable()
```

Also, I don't understand your goal here.
The code below works with all Quarto features.

---
title: "Quarto Playground"
format: html
embed-resources: true
---

This is a playground for Quarto.

```{python}
#| echo: true
import panel as pn

pn.extension(design="material")

slider = pn.widgets.IntSlider(name="Select a value", value=10, start=0, end=100)
pn.Column(pn.rx("You selected: {}").format(slider),).servable()
```

```{python}
# Import necessary libraries
import holoviews as hv
from holoviews import opts
hv.extension('bokeh')

# Create some data
data = {
    'x': [1, 2, 3, 4, 5],
    'y': [5, 6, 7, 8, 9]
}

# Create a scatter plot
scatter = hv.Scatter(data, 'x', 'y')

# Set some options
scatter.opts(opts.Scatter(width=700, size=10, color='red'))

# Display the plot
scatter
```

[MarcSkovMadsen]

Thanks for the questions.

I don't think I need to define a new kernel. Because with Pyscript, Pyodide and Panel the rendering can take place when the user opens the document. And the output will be live and interactive.

What I want is to take a code block as input and output the code nicely styled on top and some custom html output below. I would like to be able to control this using the exact same execution options as the {python} filter.

The goal is to get live, interactive output powered by PyScript, Pyodide and/ or Panel. Not static output created by Jupyter.

The examples below have the output rendered by PyScript/ Panel instead of Jupyter.

PyScript

Link: https://awesome-panel.github.io/holoviz-quarto/example-pyscript.html

In this example I am able to render using pyscript instead of Jupyter.

<div id="out-1" class="lds-dual-ring pn-container"></div>
<script type="py">
import panel as pn

slider = pn.widgets.IntSlider(name="Select a value", value=10, start=0, end=100)
pn.Column(
    pn.rx("You selected: {}").format(slider),
).servable(target="out-1")
</script>

I just want to be able to write this as a nice code block with a custom {pyscript-python} filter.

```{pyscript-python}
import panel as pn

slider = pn.widgets.IntSlider(name="Select a value", value=10, start=0, end=100)
pn.Column(
    pn.rx("You selected: {}").format(slider),
)
```

I want this filter to have the same execution options as the {python} filter such that I can echo or not echo the code. I can code-fold it. But I hope and imagine its very easy to do as I can reuse most of the code that powers the {python} filter. I had hoped it would very easy to create the filter because most of the code is already there in Quarto. I just don't know where to find it.

I try to explain this in the video below.

explanation.mp4

Panel Convert

Link: https://awesome-panel.github.io/holoviz-quarto/example-panel-convert-python.html

Here I have created a custom filter {panel-convert-python} that can convert the code of a code block into an iframe with the code to run Panel apps live and interactively in the browser. I just don't know how to easily add the code section on top of the iframe and control it with echo and code-fold execution options.

[mcanouil]

If you want add classes or attributes, then you can use any filter and do whatever you want.
There is no need to use some kind of custom code cell language.

Anyhow, you can get inspiration from:

• https://github.com/quarto-ext/shinylive
• https://github.com/coatless/quarto-webr

[coatless]

@MarcSkovMadsen cool use case! I've been playing around recently with a direct Pyodide implementation that doesn't have sliders/widgets; but, exists off a single pyodide thread/download without the iframe encapsulation.

With the proposed panel-convert-python custom code cell, you're entering territory where "here be dragons". Some initial thoughts regarding custom code cells were discussed in #4839 and #4761. The current approach is to handle options inside the extension without a direct link from Quarto. Thus, there are no official APIs for working with custom cells and/or extracting options.

That said, the approach taken by the two quarto extensions mentioned by @mcanouil handle option processing in slightly different ways:

• shinylive passes the extraction by moving it from Lua to a deno parse script that calls another script to extract the options.

• quarto-webr attempts to parse cell options in Lua and, in the next version, move the cell options into the document so that the appropriate action can be taken (c.f. #117 for proposed list). The quarto-pyodide extension I'm working on will also likely take this approach.

So, you can likely get the echo option working by adding a code block above the dynamic output. For the code-folding option, Quarto is wrapping the code cell) with:

<details>
<summary>Code</summary>

${codeTxtHere}

</details>

I know the above is not ideal; but hopefully, that can help you out while we await for 1.5 release (sometime in 2024)

[mcanouil]

Thanks @coatless for shimming in!

[MarcSkovMadsen]

Also thanks from me @coatless . It gives me some things to study. Some quick comments.

PyScript is getting to the place where it might be useful for you to consider as an alternative to Pyodide. It will provide most of the functionality you will need out of the box. Including an editor. I like the style of your editor more than the one of pyscript though. But it might be useful just to chime in on the PyScript github describing your feature requests and needs. It will help guide the PyScript editor. You don't have to use it. But it will just help make the pyscript editor better for everyone.

It might also be useful to think your extension into a broader context of technical documentation in general like Sphinx and mkDocs.

That is my goal for the HoloViz experiment. To develop something that can be used across those 3 frameworks. We already have a sphinx extension https://panel.holoviz.org/how_to/wasm/sphinx.html that should take the next step some day.

I am also considering one day to make a {panel-live} extension similar to the {shiny-live} extension. The tabbed editor with the possibility to add new document is quite powerful. My understanding is that the {shiny-live} extension only works for shiny apps. Not for the Jupyter and HoloViz ecosystems without Shiny. You need to start the shiny app server? If someone has a link to the architecture of Shiny Live, it would be really useful.

[coatless]

@MarcSkovMadsen thanks for the heads up on pyscript! I'm more from the R world, so that was not registered on my radar. (Though, I think I had tunnel vision with pyodide after having a glimpse of {voici} and {voila}.)

On the note of technical documentation, I am eager to start moving in that direction. I can say that there are two interesting Quarto extensions targeting package documentation:

• {quartodocs} (repo): Converts a Python Package's docstrings over to a Quarto website similar to Sphinx
• {altdocs} (repo): Alternative to R's {pkgdown} that generates a documentation website using either Quarto, Docsify, MkDocs, or Docute

I think one worry that I have is the WASM binary build for development packages in R/Python might be slightly problematic. Only recently was there a re-usable workflow developed for R WASM packages.

Regarding the {shinylive-python}/{shinylive-r} editor details, be prepared to code dive a bit inside the main shinylive repository. The Quarto Extension for Shinylive ({quarto-ext/shinylive}) facilitates interacting with the shinylive assets found in the main repo. You can trigger a local developer environment by running make serve. The editor is built using React. You can view its source here: src/Components/Editor.tsx. Lastly, I'm not aware of any public-facing development document that describes shinylive's architecture outside of the Shinylive: Shiny + WebAssembly
post.

---

Jumping back over to your documentation example, I noticed on the page before there is a quick example on pyscript. Is the py-config and py-script listed portions all that would be required for the panel extension?

https://panel.holoviz.org/how_to/wasm/standalone.html#pyscript

[MarcSkovMadsen]

Yes. (Regarding What is needed for Panel)