Julia inline code

[asalber]

How can I run Julia code inline?
I have tried `j 1 + 2` as it does Weave.jl but it doesn't work.

[jjallaire]

There is currently no support for inline code in Jupyter. We have been waiting for a "standard" to emerge for Jupyter inline code (see this discussion: https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525).

[hebertodelrio]

If you are using quarto qmd files, I believe this behavior can be achieved using Notebook Filters. The following filter will look for {{.....}} in a Markdown cell and replace it with the corresponding value. Inside the mustaches can be anything that can be evaluated in a code cell. To achieve this we use two preprocessor.

You need to add in the metadata the following line:

ipynb-filters:
  - inline_code.py

and copy in the same directory that your qmd file is located the following code (save it as inline_code.py):

import re
import sys

import nbformat
from nbconvert.preprocessors import ExecutePreprocessor
from nbconvert.preprocessors import Preprocessor



class ExecuteCodeMarkdownPreprocessor(ExecutePreprocessor):

    def __init__(self, **kw):
        self.sections = {'default': True} # maps section ID to true or false
        self.EmptyCell = nbformat.v4.nbbase.new_raw_cell("")

        super().__init__(**kw)

    def preprocess_cell(self, cell, resources, cell_index, store_history=True):
        """
        Executes a single code cell. See base.py for details.
        To execute all cells see :meth:`preprocess`.
        """

        if cell.cell_type not in ['code', 'markdown']:
            return cell, resources

        if cell.cell_type == 'code':
            # Do code stuff
            return self.preprocess_code_cell(cell, resources, cell_index, store_history)

        elif cell.cell_type == 'markdown':
            # Do markdown stuff
            cell, resources = self.preprocess_markdown_cell(cell, resources, cell_index, store_history)
            return cell, resources
        else:
            # Don't do anything
            return cell, resources

    def preprocess_code_cell(self, cell, resources, cell_index, store_history):
        """ Process code cell. Follow preprocess_cell from ExecutePreprocessor
        """
        self._check_assign_resources(resources)
        cell = self.execute_cell(cell, cell_index, store_history=True)
        return cell, self.resources

    def preprocess_markdown_cell(self, cell, resources, cell_index, store_history):
        # Find and execute snippets of code
        cell['metadata']['variables'] = {}
        for m in re.finditer("{{(.*?)}}", cell.source):
            # Execute code
            self.nb.cells.append(nbformat.v4.nbbase.new_code_cell(m.group(1)))
            fakecell, resources = self.preprocess_code_cell(self.nb.cells[-1], resources, len(self.nb.cells)-1, store_history)
            self.nb.cells.pop()
            # Output found in cell.outputs
            # Put output in cell['metadata']['variables']
            for output in fakecell.outputs:
                html = self.convert_output_to_html(output)
                if html is not None:
                    cell['metadata']['variables'][fakecell.source] = html
                    break
        return cell, resources

    def convert_output_to_html(self, output):
        """Convert IOpub output to HTML

        See https://github.com/ipython-contrib/IPython-notebook-extensions/blob/master/nbextensions/usability/python-markdown/main.js
        """
        if output['output_type'] == 'error':
            text = '**' + output.ename + '**: ' + output.evalue
            return text
        elif output.output_type == 'execute_result' or output.output_type == 'display_data':
            data = output.data
            if 'text/latex' in data:
                html = data['text/latex']
                return html
            elif 'image/svg+xml' in data:
                # Not supported
                #var svg = ul['image/svg+xml'];
                #/* embed SVG in an <img> tag, still get eaten by sanitizer... */
                #svg = btoa(svg);
                #html = '<img src="data:image/svg+xml;base64,' + svg + '"/>';
                return None
            elif 'image/jpeg' in data:
                jpeg = data['image/jpeg']
                html = '<img src="data:image/jpeg;base64,' + jpeg + '"/>'
                return html
            elif 'image/png' in data:
                png = data['image/png']
                html = '<img src="data:image/png;base64,' + png + '"/>'
                return html
            elif 'text/markdown' in data:
                text = data['text/markdown']
                return text
            elif 'text/html' in data:
                html = data['text/html']
                return html
            elif 'text/plain' in data:
                text = data['text/plain']
                # Strip <p> and </p> tags
                # Strip quotes
                # html.match(/<p>([\s\S]*?)<\/p>/)[1]
                text = re.sub(r'<p>([\s\S]*?)<\/p>', r'\1', text)
                text = re.sub(r"'([\s\S]*?)'",r'\1', text)
                return text
            else:
            # Some tag we don't support
                return None
        else:
            return None



class PyMarkdownPreprocessor(Preprocessor):
    """
    :mod:`nbconvert` Preprocessor for the python-markdown nbextension.

    This :class:`~nbconvert.preprocessors.Preprocessor` replaces kernel code in
    markdown cells with the results stored in the cell metadata.
    """

    def replace_variables(self, source, variables):
        """
        Replace {{variablename}} with stored value
        """
        try:
            replaced = re.sub(
                "{{(.*?)}}", lambda m: variables.get(m.group(1), ''), source)
        except TypeError:
            replaced = source
        return replaced

    def preprocess_cell(self, cell, resources, index):
        """
        Preprocess cell

        Parameters
        ----------
        cell : NotebookNode cell
            Notebook cell being processed
        resources : dictionary
            Additional resources used in the conversion process.  Allows
            preprocessors to pass variables into the Jinja engine.
        cell_index : int
            Index of the cell being processed (see base.py)
        """
        if cell.cell_type == "markdown":
            if hasattr(cell['metadata'], 'variables'):
                variables = cell['metadata']['variables']
                if len(variables) > 0:
                    cell.source = self.replace_variables(
                        cell.source, variables)
        return cell, resources



nb = nbformat.reads(sys.stdin.read(), as_version = 4)
ep = ExecuteCodeMarkdownPreprocessor(timeout=600)
ep.preprocess(nb, {})
pymk = PyMarkdownPreprocessor()
pymk.preprocess(nb, {})
nbformat.write(nb, sys.stdout)

[Lightup1]

It should not happen.
In my machine (windows 11), it just works well with TeXLive and julia. Julia seems to have no reason to interfere the tlmgr.
Have you installed julia kernels via ] add IJulia; build IJulia.

[hebertodelrio]

Sometimes in Windows machines, the user that is running all the programs is the administrator so it has all privileges, if that is your case you may not notice that quarto is actually running tlmgr and updating texlive (or it did the first time that need packages to be updated).

I am running OS X, although my user has sudo privileges (admin) but in order to do that the command needs to be prepended with sudo (which I did when I ran quarto preview juliatest.qmd) but without any positive result.

I installed Julia through conda In an environment in which I am testing this, quarto has been installed twice, the first time only for my user (installed in the home directory of the user) and then it was removed and installed globally (all user have aces to it) with the same result. Once Julia was installed, I fired it up and did

using Pkg
Pkd.add("IJulia")
Pkg.add("Revise")

I can have a Jupiter notebook with a Julia kernel after that, which makes me believe that it was installed correctly.

The problem is with quarto attempting to build a PDF (when xelatex is being used or any other tex-engine which I have tried aswell). In all these cases tlmgr is attempting to update some packages (but it is not reported which packages) and the updating is failing.

[Lightup1]

It seems worth to raise an issue on your Julia case I think. 😥

[hebertodelrio]

I did raise an issue, since this is not a jupyter, Python, Julia, R nor RStudio issue. I tested against all of them and the behavior is the same. Quarto is attempting to update TeX Live distribution and it is failing.

Later in the week I will try on Windows to see if this is an OS issue or more general.

[Dijkie85]

import re
import sys

import nbformat
from nbconvert.preprocessors import ExecutePreprocessor
from nbconvert.preprocessors import Preprocessor

class ExecuteCodeMarkdownPreprocessor(ExecutePreprocessor):

def __init__(self, **kw):
    self.sections = {'default': True} # maps section ID to true or false
    self.EmptyCell = nbformat.v4.nbbase.new_raw_cell("")

    super().__init__(**kw)

def preprocess_cell(self, cell, resources, cell_index, store_history=True):
    """
    Executes a single code cell. See base.py for details.
    To execute all cells see :meth:`preprocess`.
    """

    if cell.cell_type not in ['code', 'markdown']:
        return cell, resources

    if cell.cell_type == 'code':
        # Do code stuff
        return self.preprocess_code_cell(cell, resources, cell_index, store_history)

    elif cell.cell_type == 'markdown':
        # Do markdown stuff
        cell, resources = self.preprocess_markdown_cell(cell, resources, cell_index, store_history)
        return cell, resources
    else:
        # Don't do anything
        return cell, resources

def preprocess_code_cell(self, cell, resources, cell_index, store_history):
    """ Process code cell. Follow preprocess_cell from ExecutePreprocessor
    """
    self._check_assign_resources(resources)
    cell = self.execute_cell(cell, cell_index, store_history=True)
    return cell, self.resources

def preprocess_markdown_cell(self, cell, resources, cell_index, store_history):
    # Find and execute snippets of code
    cell['metadata']['variables'] = {}
    for m in re.finditer("{{(.*?)}}", cell.source):
        # Execute code
        self.nb.cells.append(nbformat.v4.nbbase.new_code_cell(m.group(1)))
        fakecell, resources = self.preprocess_code_cell(self.nb.cells[-1], resources, len(self.nb.cells)-1, store_history)
        self.nb.cells.pop()
        # Output found in cell.outputs
        # Put output in cell['metadata']['variables']
        for output in fakecell.outputs:
            html = self.convert_output_to_html(output)
            if html is not None:
                cell['metadata']['variables'][fakecell.source] = html
                break
    return cell, resources

def convert_output_to_html(self, output):
    """Convert IOpub output to HTML

    See https://github.com/ipython-contrib/IPython-notebook-extensions/blob/master/nbextensions/usability/python-markdown/main.js
    """
    if output['output_type'] == 'error':
        text = '**' + output.ename + '**: ' + output.evalue
        return text
    elif output.output_type == 'execute_result' or output.output_type == 'display_data':
        data = output.data
        if 'text/latex' in data:
            html = data['text/latex']
            return html
        elif 'image/svg+xml' in data:
            # Not supported
            #var svg = ul['image/svg+xml'];
            #/* embed SVG in an <img> tag, still get eaten by sanitizer... */
            #svg = btoa(svg);
            #html = '<img src="data:image/svg+xml;base64,' + svg + '"/>';
            return None
        elif 'image/jpeg' in data:
            jpeg = data['image/jpeg']
            html = '<img src="data:image/jpeg;base64,' + jpeg + '"/>'
            return html
        elif 'image/png' in data:
            png = data['image/png']
            html = '<img src="data:image/png;base64,' + png + '"/>'
            return html
        elif 'text/markdown' in data:
            text = data['text/markdown']
            return text
        elif 'text/html' in data:
            html = data['text/html']
            return html
        elif 'text/plain' in data:
            text = data['text/plain']
            # Strip <p> and </p> tags
            # Strip quotes
            # html.match(/<p>([\s\S]*?)<\/p>/)[1]
            text = re.sub(r'<p>([\s\S]*?)<\/p>', r'\1', text)
            text = re.sub(r"'([\s\S]*?)'",r'\1', text)
            return text
        else:
        # Some tag we don't support
            return None
    else:
        return None

class PyMarkdownPreprocessor(Preprocessor):
"""
:mod:nbconvert Preprocessor for the python-markdown nbextension.

This :class:`~nbconvert.preprocessors.Preprocessor` replaces kernel code in
markdown cells with the results stored in the cell metadata.
"""

def replace_variables(self, source, variables):
    """
    Replace {{variablename}} with stored value
    """
    try:
        replaced = re.sub(
            "{{(.*?)}}", lambda m: variables.get(m.group(1), ''), source)
    except TypeError:
        replaced = source
    return replaced

def preprocess_cell(self, cell, resources, index):
    """
    Preprocess cell

    Parameters
    ----------
    cell : NotebookNode cell
        Notebook cell being processed
    resources : dictionary
        Additional resources used in the conversion process.  Allows
        preprocessors to pass variables into the Jinja engine.
    cell_index : int
        Index of the cell being processed (see base.py)
    """
    if cell.cell_type == "markdown":
        if hasattr(cell['metadata'], 'variables'):
            variables = cell['metadata']['variables']
            if len(variables) > 0:
                cell.source = self.replace_variables(
                    cell.source, variables)
    return cell, resources

nb = nbformat.reads(sys.stdin.read(), as_version = 4)
ep = ExecuteCodeMarkdownPreprocessor(timeout=600)
ep.preprocess(nb, {})
pymk = PyMarkdownPreprocessor()
pymk.preprocess(nb, {})
nbformat.write(nb, sys.stdout)

HI! Is this working? I tried it with a Julia 1.8 kernel and I don't think it even loaded the filter...

[lyon-fnal]

It's not really inline, but you can kinda fake it with something like the following ...

```{julia}
using Markdown

md"This is some **markdown** text. The square root of two is $(sqrt(2))"
```

It will look like regular text in the document.

For example, I have a presentation that has a code block ...

```{julia}
using Markdown
using Printf

totalGB = sum(df_big.size)/(1024^3)
totalMin = sum(df_big.s)/60
avgMBPerSec = totalGB*1024/(totalMin*60)
md"Transferred $(@sprintf(\"%d\", totalGB)) GB in $(@sprintf(\"%d\", totalMin)) minutes with three other simultaneous transfers going<br>Averaged $(@sprintf(\"%d\", avgMBPerSec)) MB/s"
```

And it looks like (see the bottom of the slide)...

As you can see, the Markdown text from the code block looks just like the regular document text.

[Dijkie85]

Though not super comfortable, this is quite a neat workaround. The functionality for reproducibility is there. Thank you!

[hebertodelrio]

The new release does not load ipynb-filters on qmd files.

…

-- Heberto del Rio

On September 14, 2022 at 5:29:29 PM, Dijkie85 ***@***.***) wrote: import re import sys import nbformat from nbconvert.preprocessors import ExecutePreprocessor from nbconvert.preprocessors import Preprocessor class ExecuteCodeMarkdownPreprocessor(ExecutePreprocessor): def __init__(self, **kw): self.sections = {'default': True} # maps section ID to true or false self.EmptyCell = nbformat.v4.nbbase.new_raw_cell("") super().__init__(**kw) def preprocess_cell(self, cell, resources, cell_index, store_history=True): """ Executes a single code cell. See base.py for details. To execute all cells see :meth:`preprocess`. """ if cell.cell_type not in ['code', 'markdown']: return cell, resources if cell.cell_type == 'code': # Do code stuff return self.preprocess_code_cell(cell, resources, cell_index, store_history) elif cell.cell_type == 'markdown': # Do markdown stuff cell, resources = self.preprocess_markdown_cell(cell, resources, cell_index, store_history) return cell, resources else: # Don't do anything return cell, resources def preprocess_code_cell(self, cell, resources, cell_index, store_history): """ Process code cell. Follow preprocess_cell from ExecutePreprocessor """ self._check_assign_resources(resources) cell = self.execute_cell(cell, cell_index, store_history=True) return cell, self.resources def preprocess_markdown_cell(self, cell, resources, cell_index, store_history): # Find and execute snippets of code cell['metadata']['variables'] = {} for m in re.finditer("{{(.*?)}}", cell.source): # Execute code self.nb.cells.append(nbformat.v4.nbbase.new_code_cell(m.group(1))) fakecell, resources = self.preprocess_code_cell(self.nb.cells[-1], resources, len(self.nb.cells)-1, store_history) self.nb.cells.pop() # Output found in cell.outputs # Put output in cell['metadata']['variables'] for output in fakecell.outputs: html = self.convert_output_to_html(output) if html is not None: cell['metadata']['variables'][fakecell.source] = html break return cell, resources def convert_output_to_html(self, output): """Convert IOpub output to HTML See https://github.com/ipython-contrib/IPython-notebook-extensions/blob/master/nbextensions/usability/python-markdown/main.js """ if output['output_type'] == 'error': text = '**' + output.ename + '**: ' + output.evalue return text elif output.output_type == 'execute_result' or output.output_type == 'display_data': data = output.data if 'text/latex' in data: html = data['text/latex'] return html elif 'image/svg+xml' in data: # Not supported #var svg = ul['image/svg+xml']; #/* embed SVG in an <img> tag, still get eaten by sanitizer... */ #svg = btoa(svg); #html = '<img src="data:image/svg+xml;base64,' + svg + '"/>'; return None elif 'image/jpeg' in data: jpeg = data['image/jpeg'] html = '<img src="data:image/jpeg;base64,' + jpeg + '"/>' return html elif 'image/png' in data: png = data['image/png'] html = '<img src="data:image/png;base64,' + png + '"/>' return html elif 'text/markdown' in data: text = data['text/markdown'] return text elif 'text/html' in data: html = data['text/html'] return html elif 'text/plain' in data: text = data['text/plain'] # Strip <p> and </p> tags # Strip quotes # html.match(/<p>([\s\S]*?)<\/p>/)[1] text = re.sub(r'<p>([\s\S]*?)<\/p>', r'\1', text) text = re.sub(r"'([\s\S]*?)'",r'\1', text) return text else: # Some tag we don't support return None else: return None class PyMarkdownPreprocessor(Preprocessor): """ :mod:nbconvert Preprocessor for the python-markdown nbextension. This :class:`~nbconvert.preprocessors.Preprocessor` replaces kernel code in markdown cells with the results stored in the cell metadata. """ def replace_variables(self, source, variables): """ Replace {{variablename}} with stored value """ try: replaced = re.sub( "{{(.*?)}}", lambda m: variables.get(m.group(1), ''), source) except TypeError: replaced = source return replaced def preprocess_cell(self, cell, resources, index): """ Preprocess cell Parameters ---------- cell : NotebookNode cell Notebook cell being processed resources : dictionary Additional resources used in the conversion process. Allows preprocessors to pass variables into the Jinja engine. cell_index : int Index of the cell being processed (see base.py) """ if cell.cell_type == "markdown": if hasattr(cell['metadata'], 'variables'): variables = cell['metadata']['variables'] if len(variables) > 0: cell.source = self.replace_variables( cell.source, variables) return cell, resources nb = nbformat.reads(sys.stdin.read(), as_version = 4) ep = ExecuteCodeMarkdownPreprocessor(timeout=600) ep.preprocess(nb, {}) pymk = PyMarkdownPreprocessor() pymk.preprocess(nb, {}) nbformat.write(nb, sys.stdout) HI! Is this working? I tried it with a Julia 1.8 kernel and I don't think it even loaded the filter... — Reply to this email directly, view it on GitHub <#1520 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI535LPNA75GWORCMIZTZTV6I7TTANCNFSM54RXWFEQ> . You are receiving this because you commented.Message ID: ***@***.***>

[cscheid]

@hebertodelrio Thanks for letting us know! can you file a github issue with a reproducible example?

[jjallaire]

That's actually by design, ipynb-filters are intended to adapt ipynb notebooks created in another context so they can be rendered by Quarto/Pandoc (e.g. they might need cells removed, special annotations removed, etc.). They aren't intended to process qmd files (which you can already do quite a bit of processing on with Lua filters).

[jjallaire]

Yes they did used to work on qmds but we made the decision to restrict them to ipynb in version 1.1.

[hebertodelrio]

I do understand. To me the ‘inline code’ problem has two faces: 1. QMD: quarto has to have a way to ‘compute’ inline code in any markdown. One possible solution that quarto has presented is to do some trick (depending on the kernel used) by turning the markdown into a code cell add some extra flags and generate markdown text. To me this can be avoided by implementing preprocessing the qmd file turn the markdown with inline code into a code cell with the appropriate trickery so jupyter can process it as suggested by quarto. By not providing a solution transparent to the user the need of ipynb filter is important in qmd (of course quarto could do all of this by modifiying its implementation a bit) 2. JUPYTER: jupyter users have had the same issue since inception, jupytelab came up with some solution, their problem resides in that they want to see the markdown turn into text with inline code evaluated ‘live’ so they have to use JavaScript. In my opinion these are two different worlds that have a Similar problem, they are not the same problem, and quarto should not need to wait for jupyter to solve their problem to be able to solve quarto’s problem. Which somehow was suggested in a thread. Sweve the older brother of quarto has solved the issue for Rmd, I do not see why quarto cannot solve this issue for qmd.

…

-- Heberto del Rio

On September 14, 2022 at 9:42:00 PM, J.J. Allaire ***@***.***) wrote: That's actually by design, ipynb-filters are intended to adapt ipynb notebooks created in another context so they can be rendered by Quarto/Pandoc (e.g. they might need cells removed, special annotations removed, etc.). They aren't intended to process qmd files (which you can already do quite a bit of processing on with Lua filters). — Reply to this email directly, view it on GitHub <#1520 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI535OQXWGFTM4AOU4IEC3V6J5GRANCNFSM54RXWFEQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[jjallaire]

Yes, I agree it would be nicer if we could do this processing on user's behalf. The reason we currently don't is somewhat tied up in the resolution of the issue for Jupyter. To start with and for those reading this thread who aren't aware, here are the two ways currently available to do "inline" execution within markdown:

1. https://quarto.org/docs/computations/execution-options.html#inline-code
2. https://quarto.org/docs/computations/execution-options.html#raw-output

Both essentially use normal code cells that in turn have mostly strings w/ markdown (often in multiline form) along with some sort of string interpolation.

While this is inelegant, it does have the positive attribute of working in both qmd files and in ipynb files using any notebook front end. This will also work if the notebook is rendered/converted outside of Quarto (e.g by nbconvert or other tools like Viola that enable you to publish rendered notebooks).

So the current suggested solution is broadly compatible. While we could create a syntax akin to what knitr has (e.g. `python x+1`) this would only work w/ Quarto and then only when rendering qmds or executing ipynbs (as if you aren't executing the ipynb then the inline code would need to have been expanded during notebook authoring time, and Jupyter would know nothing of the Quarto specific syntax).

Our preference would be that this discussion (https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525) reaches a consensus that is then supported by Jupyter, Jupyter Lab, nbconvert, nbclient, etc. At that point we'd have both elegance and compatibility. In the meantime we are leaning towards compatibility over elegance.

[cderv]

FWIW There has been a recent addition for this in 1.4 : #6190

[Dijkie85]

This is huge!