Quarto inserts random backslashes in visual mode editor

[eteitelbaum]

When I am editing in visual mode, I am noticing that occasionally Quarto is inserting random backslashes into the text. These don't show up in the visual mode editor but I can see them when I switch over to the source mode. So here is an example:

I wrote:

($y$%)

But Quarto turned it into:

(\$y\$%)

I also notice this happening when I am cutting and pasting stuff from an older R Markdown document. If I am in visual mode and I try to past a bunch of stuff I see random forward slashes in front of every line. I don't have a reproducible example of this, just something I have experienced in the past.

Are these supposed to be escape characters? Why is Quarto inserting them?

[jjallaire]

Those slashes are added by Pandoc when we round trip the source through its internal representation. Backslashes in markdown simply ensure that the characters are not interpreted as e.g. an equation.

[jjallaire]

No, they won't be added if it's a valid LaTeX equation. Note that it is Pandoc that is adding these based on it determine via its internal parsing heuristics that this is not an equation).

I don't know what version of RStudio you are running but we did fix a number of bugs related to Pandoc escaping in the Visual Editor in the most recent release (2022.07) so you should definitely be sure you are running that.

You should file a defect if you have unexpected output not unexpected markdown (as you can generally trust all of the markdown transformations that Pandoc makes because these are the identical transformations that it applies internally when rendering.

[eteitelbaum]

Yes I am running 2022.07.01. I am not sure what a "valid LaTeX equation" is but it is something I have experienced intermittently with the most simple variables or symbols like $y$ or $\beta$ and almost every equation I insert that I used to run in R Markdown.

For example, if I have markdown code:

\begin{equation}
WTP = \frac {\beta_f}  {\beta_p}
\end{equation}

and I transform it to

$$
WTP = \frac {\beta_f}  {\beta_p}
$$

Quarto will stick slashes before the $ symbols.

Like I said it is also adding an additional slash before LaTeX commands like \noindent and also back ticks for package names and inline code like if I write "In this analysis I used the rhierMnlRwMixture routine from the bayesm package" it is inserting slashes before the name of the function and the package.

[jjallaire]

for some reason math is not being recognized at all -- perhaps you are targeting a markdown format with no support for math?

I'm not able to reproduce the behavior you are describing. The best approach from here is for you to create a minimal reproducible example (complete source code of a qmd that exhibits the behavior) along with a screenshot of the rendered output you find problematic.

[eteitelbaum]

I am not sure what you mean by "perhaps you are targeting a markdown format with no support for math." I am editing a Quarto .pdf document in the visual editor mode. Is there some special markdown format I need to be aware of?

It just happened again. This time I was editing a website and every time I try to insert a hyperlink Quarto inserts a backslash before the opening and closing brackets like this \["Importing Data Into Excel Spreadsheets"\]. Here is the code. Also attaching the screen shot:

---
title: "Course Schedule"
tbl-colwidths: [5,10,30,30,5,5,5,5,5]
---

This page contains an outline of the topics, content, and assignments for the semester. Note that this schedule will be updated as the semester progresses, with all changes documented here.
<img width="926" alt="cpe-website-malfunction" src="https://user-images.githubusercontent.com/62634082/185527053-c41529a2-8b2b-44b9-95db-8fc5afd11184.png">


::: column-screen-inset-right
| Week | Date | Topic                                 | Skill                      | Read              | Dataquest                                                                                                                                         | Quiz | Exam | Project |
|--------|--------|--------|--------|--------|--------|--------|--------|--------|
| 1    |      | Course overview                       | Explore MS 365 & Dataquest |                   |                                                                                                                                                   |      |      |         |
|      |      | The economy and society               | Understanding data         |                   | ["Defining Data"](https://app.dataquest.io/c/141/m/756/defining-data/1/introduction-to-data-foundations)                                          |      |      |         |
| 2    |      | Institutions and development          | Using data                 |                   | ["How to Use Data"](https://app.dataquest.io/c/141/m/757/how-to-use-data/1/how-to-use-data)                                                       |      |      |         |
|      |      | Modernization theory                  | Analyzing data             |                   | ["The Data Analysis Process"](https://app.dataquest.io/c/141/m/758/the-data-analysis-process/1/the-data-analysis-process)                         |      |      |         |
| 3    |      | Democratic backsliding                | Importing data             |                   | \["Importing Data Into Excel Spreadsheets"\](https://app.dataquest.io/c/127/m/652/importing-data-into-excel-spreadsheets/1/welcome-to-the-course) |      |      |         |
|      |      | The resource curse                    | Organizing data            |                   | "Organizing Data"                                                                                                                                 |      |      |         |
| 4    |      | Women's empowerment                   | Cleaning data              |                   | "Cleaning Data Parts I & II"                                                                                                                      |      |      |         |
|      |      | Exam 1 review                         |                            |                   |                                                                                                                                                   |      |      |         |
| 5    |      | Clientelism                           | Bar charts                 |                   | "Pie, Column & Histogram Charts"                                                                                                                  |      |      |         |
|      |      | Ethnicity & service provisioning      | Line charts                |                   | "Line, Scatter & Combo Charts"                                                                                                                    |      |      |         |
| 6    |      | Welfare states in the OECD            | Scatter plots              | Whither the U.S.? | "Line, Scatter & Combo Charts"                                                                                                                    |      |      |         |
|      |      | Welfare states in LMICs               | Maps                       |                   |                                                                                                                                                   |      |      |         |
| 7    |      | Neoliberalism                         | Intro to Zotero            |                   |                                                                                                                                                   |      |      |         |
|      |      | Crisis and reform                     | Using Zotero               |                   |                                                                                                                                                   |      |      |         |
| 8    |      | Globalization (focus on china shock?) | Topic selection            |                   |                                                                                                                                                   |      |      |         |
|      |      | Inequality                            | Sources and data           |                   |                                                                                                                                                   |      |      |         |
| 9    |      | The populist backlash                 | Writing an outline         |                   |                                                                                                                                                   |      |      |         |
|      |      | Exam 2 review                         |                            |                   |                                                                                                                                                   |      |      |         |
| 10   |      | Unions and collective bargaining      |                            |                   |                                                                                                                                                   |      |      |         |
|      |      | Varieties of capitalism               |                            |                   |                                                                                                                                                   |      |      |         |
| 11   |      | The great resignation                 |                            |                   |                                                                                                                                                   |      |      |         |
|      |      | Automation & universal basic income   |                            |                   |                                                                                                                                                   |      |      |         |
| 12   |      | Political consumerism                 |                            |                   |                                                                                                                                                   |      |      |         |
|      |      | Environmental and social governance   |                            |                   |                                                                                                                                                   |      |      |         |
| 13   |      | Final project workshop (small groups) |                            |                   |                                                                                                                                                   |      |      |         |
|      |      | No class-Thanksgiving break           |                            |                   |                                                                                                                                                   |      |      |         |
| 14   |      | Cryptocurrencies                      |                            |                   |                                                                                                                                                   |      |      |         |
|      |      | Defi and DAOs                         |                            |                   |                                                                                                                                                   |      |      |         |
| 15   |      | Extended reality                      |                            |                   |                                                                                                                                                   |      |      |         |
|      |      | Exam 3 review                         |                            |                   |                                                                                                                                                   |      |      |         |
:::

[eteitelbaum]

P.S. I wanted to clarify that it is only happening in the visual editor. It does not happen when I just stick to editing in source mode.

[jjallaire]

I'm not sure what's happening in your local environment but I am not able to reproduce any of this with the latest version of RStudio IDE. Note that I modified your source code to remove the escaping for [ and ] (being sure to add a couple spaces after the link so the table is preserved) then subsequently round-tripped it through the visual editor w/ no problems.

This is BTW an RStudio IDE issue not a quarto-cli issue so you should report it there. If you can provide the absolute minimum qmd document that exhibits the problem that will be extremely helpful. You should provide the source to a document that loads correctly in the visual editor (i.e. has no escaping) which then consistently has incorrect escaping added on transforming back to source.

[eteitelbaum]

It's OK. I'll just stick to the source editor for now. Maybe it will go away on its own with a future update.

I will note that I have experienced the issue on two separate PCs both running the most recent version of R Studio. But it is something that occurs intermittently and not something that I can reproduce in a way that someone could easily troubleshoot. I just thought I'd check to see if other people were having the same issue.

Thanks for your help.

[eeholmes]

@eteitelbaum This is definitely a thing. I work with LaTeX with the visual editor and have run into this too. In fact I was warned by a colleague "Watch out, the visual editor trashes LaTeX." For work, I am careful to stick with the source editor, but for teaching Quarto I do want to use the visual editor.

My most common issue happens when I don't have a blank space between text and display math.

Then I head to the visual editor for a moment. Still looks good.

And then back to source, and it trashed my LaTeX. :( Not only this equation....but every other valid $$....$$ equation in the document because that trailing $$ is interpreted as the start a display math environment. But it's invalid so every following display math via $$ is broken.

If I had a blank line between the text and display math, it won't happen. But LaTeX users remove the blank lines so that LaTeX doesn't interpret lines after an equation as a new paragraph and add indentation + extra spacing.

In general though, I have personally found that Pandoc is too destructive to $$ .. $$. If you look at the TeX is generates, it added random extra blank lines before the equation, thus causing LaTeX to interpret the equation and next sentences as new paragraphs (with all the indentation and par spacing that comes with that).

So I tend to use \begin{equation}...\end{equation}, but here too the visual editor changes the source.

Before switching to visual editor:

After switching the visual and then back to source:

Ok not nice, but it does still make valid TeX unlike what happens to $$...$$. It becomes this in the .tex file:

I am not happy that it changed $a$ to \(a\) but this is still valid tex. Also sadly the visual editor (nor previewer in source) does not recognize \begin{equation}...\end{equation} as display math that can be interpreted like $$ ... $$ and displayed. So I end up getting equations prepared with $$ ... $$ where I have live preview and then once I am done, I change to \begin{equation}...\end{equation} to protect the equation from Pandoc (filters I think).

[eeholmes]

P.S. to see the par indent effects, add

    include-in-header: 
      text: |
        \setlength{\parindent}{10pt}

to the pdf format. Pandoc really dislikes par indent and sets it to 0 for all non-KOMA document classes. Very strange. Or perhaps Quarto is using a modified Pandoc template.tex. I didn't check.

[eteitelbaum]

@eeholmes Thanks for articulating this issue so well.

I notice that if I write

\begin{equation}
a+b
\end{equation}

in the source editor, switch back to the visual editor and then back to the source editor, I see that the LaTeX equation is wrapped in a code chunk:

```{=tex}
\begin{equation}
a=b
\end{equation}

I had not noticed this behavior before, but it made me wonder with the LaTeX code could be "protected" from Pandoc by wrapping it in a tex code chunk in the source editor.

Also, I have noticed that the extra backslash issue always happens when I am cutting and pasting LaTeX code into the visual editor. So, for example, If I cut and paste

\begin{equation}
a=b
\end{equation}

into the visual editor, and then switch over to the source editor, I see:

\\\begin{equation}

a=b

\\\end{equation}

So I think that makes three conditions under which the extra backslashes are added:

1. When there is no space between text and the LaTeX equation;
2. When switching from the source editor to the visual editor and back again;
3. When pasting LaTeX code into the visual editor

You mentioned sticking to the source editor. I am finding it even easier to edit in VS Code with the Quarto extension.

[eeholmes]

Thanks! I have to try out VS Code with the Quarto extension. I do love the visual editor for the / shortcut for inserting stuff and especially that I can use @ and it will find all my citation references for me (! amazing). I love the image insertion feature too. So much easier to copy and paste images in when I am prepping lectures.

But I am a little scared to open up my {bookdown} Rmd books that are heavy on LaTeX equations (50-100 equations). I really wish that the visual editor would warn before altering my LaTeX documents for me and give me a chance to abort.

Re copying into the visual editor. Yeah I have also run into trouble with that too.