Future development of escaping variable values

[Taitava]

Closing an old issue #11 and moving future escaping discussion here

Issue #11 has discussed about escaping variable values. First off, as the issue is older than the Discussions section in this GitHub repository, it contains discussion about more things than just what the first version of variable value escaping will be bringing. Also, when I started the issue, I had no clue how the escaping will be done, so the issue contains varying ideas on how to implement escaping, but most of them are now obsolete.

Future coming version 0.7.0 will close issue #11. Now the situation is that an expiremental way to escape variable values has been found and has been implemented, and will be released in 0.7.0, so the old issue with partly hypothetic ideas can be closed soon.

I opened this discussion so that I can bury most of the old ideas in #11, but also so that I can pick up one idea that I might want to bring into life later. But before that, I'll revise what is the current status with how the escaping will work.

The chosen method for escaping

0.7.0 will introduce the following escaping style:

• Put a backslash \ or a ` character in front of every character that is not an ASCII letter, a number or an underscore.
• Escaping is only applied to variable values i.e. the substitution of {{variable}} expressions. Everytinhg outside of {{variables}} are not escaped, the user is expected to do any needed escaping outside of variables.
• Variable value escaping can be prevented by adding an esclamation mark ! to a variable, e.g. {{!file_name}}. Preventing escaping is not recommended, and it shold not be needed in most cases when using variables, but it's possible if a user happens to have a really rare use-case that cannot be implemented with escaping.
• I originally mentioned this method here: Variable values should be escaped #11 (comment)

For the future: URL encoding

I had already forgotten this idea. Anyway, it's a good thing not to implement this yet in 0.7.0, as it's a bit different from the basic escaping feature. But it's definitely something that will be useful.

A quote from issue #11:

Sometimes URLs might be used in commands, for example with Obsidian URI or when launching a web browser to open a specific web page or service, such as search from the internet using keywords from a variable.

For URL encoding, there are two different options regarding where the variable value will be actually placed in a URL:

* URI query parameter values, e.g. `https://duckduckgo.com/?q={{selection}}`. Here `{{selection}}` should be encoded according to [application/x-www-form-urlencoded](http://www.w3.org/TR/html4/interact/forms.html#h-17.13.4.1) (space is encoded with +).

* Everything else, e.g. `https://github.com/Taitava/{{selection}}` (to go to a specific repository under my GitHub account). Here `{{selection}}` should be encoded according to the plain [Percent-Encoding](http://tools.ietf.org/html/rfc3986#section-2.1) (space is encoded with %20).

Major parts of the above text are copied 2021-09-16 from https://stackoverflow.com/a/4744917/2754026 . The source talks about URL encoding in PHP, but the same principles can be applied to development in JavaScript/TypeScript too. I just need to find the corresponding encoding JS functions.

[Taitava]

HTML encoding

As Obsidian uses Markdown, and Markdown allows embedding HTML* in it, it woul be quite natural to have a possibility to encode any possible HTML special characters a variable value might have, e.g. <, >, /, & and ;, just to name a few.

(This kind of escaping can be needed if the variable's source can be external, e.g. {{clipboard}}. But for a variable whose source is a Markdown note, no HTML encoding is needed, e.g. {{selection}}, because if the source value would need encoding/escaping, a user would have done it already. So, this kind of encoding requires user discretion.)

Possible syntax for encoding:

• {{html?clipboard}}: Do html encoding, and also use normal shell escaping, like described in the The chosen method for escaping section above.
• {{!html?clipboard}}: Do html encoding, but not the normal shell escaping.
• {{url1?clipboard}}: Do URL encoding, and the normal shell escaping.
• {{!url1?clipboard}}: Do URL encoding, but not the normal shell escaping.
• This html? / url1? definition might still change, I'm not yet sure.
• url1 and url2 will denote different encoding styles, like mentioned in the opening message of this discussion. I don't actually like the numbers 1 and 2 there, I might try to figure out something else.

*) Actually, Markdown is compiled to HTML, and if any HTML is in the source text, Markdown does not deny it in any way, it let's the HTML pass to the final result.