[Discussion]: Inline macro with `\macro` style

[boltlessengineer]

I'm opening this issue as discussion instead of suggestion because it is incomplete proposal. I want to share some idea to implement inline macro.

I know, it smells like LaTeX.

Idea

Inline macro syntax needs macro identifier and list of arguments.
We already have good way to represent key-value arguments inline with attached modifier extensions; (key value;key;key). So we only need syntax to represent macro identifier.

In v1 spec, escaped normal character (not whitespace nor punctuation) doesn't have any additional meanings.
We can use that.

\macro-name(key value;params)

As some inline macros might need raw markup text as an argument, we an provide that with existing [...] grammar:

\macro-name[*markup* argument](key value;params)

I'm not sure if [...] should be free-form or not.

If it is free-form, we can replace $...$ syntax with this:

\latex[c = \sqrt{a^2 + b^2}]
\typst[c = sqrt(a^2 + b^2)]

If it is not free-form, we can nest the inline macros:

\my-macro[some text with \emoji(thinking)]

2nd case also shares same syntax with anchor

Example

\u(1f600) -> (unicode 1f600)

\gh[neovim/neovim] -> {https://github.com/neovim/neovim}[neovim/neovim]

\nvimhelp[:h 'shiftwidth'] -> {https://neovim.io/doc/user/options.html#'shiftwidth'}[`:h 'shiftwidth'`]

\latex[\alpha + \beta = \gamma] -> (Abstract Object representing LaTeX math equiation)

Cons

• While \u is ok, \t or \n might confuse users.
• How to handle _\m_ or -\m-? Macro name can conflict with underline/strikethrough closing modifiers.
  • This wouldn't be a problem with [Suggestion]: Generalize link-modifier to work everywhere #49

[boltlessengineer]

Another potential use-case of this syntax is citations.

\u(username)

Similar to u/username from reddit. We need to change the name to not conflict with unicode example above.

[boltlessengineer]

Some more examples of possible inline-macro use-cases

• Youtube video timestamps
• Emojis (commonly used as :smile: syntax)
• Github's auto-link for issues, PRs, discussions

These are all common cases where the Layer 1 app might want custom syntax other than simple citations. We can make 1st class citation syntax extensible, but then what's the difference from inline macros?

[vhyrro]

Few questions/remarks.

• How do you intend to escape characters after this new change is in place?
• The idea to differentiate markup from raw content is sensible in a vacuum, but it crosses over with the other tag types (carryover etc.). What happens to those?
• Following that thought, how is:

  \macro-name[
  many paragraphs
  
  contained within
  ]
  

  expected to be handled?
• How is macro expansion expected to be handled when macros return multiple paragraphs of content, and the macro invocation itself spans multiple paragraphs (like above)?

The fragility of paragraph boundaries is why we initially gave up on the idea of having parameters. We'd either have to embrace a proto-janet syntax like (function-name arg1 arg2), which can't be done as it interferes with regular markup, or we'd have to do what typst and friends do and introduce a special region where you could evaluate macros inline, like the unofficial <...> proposal in Norg syntax :)

[boltlessengineer]

How do you intend to escape characters after this new change is in place?

I'm now defining "escape sequence" to "\ + (whitespace | punctuation)" instead of previous "\ + (any one character)". Only whitespace and punctuation has special meaning in Norg syntax. We don't need to escape others.

The idea to differentiate markup from raw content is sensible in a vacuum, but it crosses over with the other tag types (carryover etc.). What happens to those?

Can you elaborate more? I couldn't understand what you are saying.

• Following that thought, how is:

  \macro-name[
  many paragraphs
  
  contained within
  ]
  

  expected to be handled?

Same to

[
many paragraphs

contained within
]

How is macro expansion expected to be handled when macros return multiple paragraphs of content, and the macro invocation itself spans multiple paragraphs (like above)?

I think this is general macro implemenation issue and not a problem specifically for inline macros.
We do need to talk about it, but not here.

---

I have some idea about paragraph boundaries issue, but in general, I think that won't be a problem as we already defined [...] syntax without an issue. I'll open a new issue dealing with it soon.

[boltlessengineer]

Actually there is an open issue about paragraph boundaries: #53
I thought I didn't open it yet.

[vhyrro]

For the same reason that #text and #(text) from #52 are way too similar, \* and \text being different things is definitely a no-go for me syntax-wise.

The idea to differentiate markup from raw content is sensible in a vacuum, but it crosses over with the other tag types (carryover etc.). What happens to those?

Can you elaborate more? I couldn't understand what you are saying.

Sorry, I mean that \macro[markup here] is equivalent to:

#macro
markup here

Which means there's significant overlap between both syntaxes. It's not much of an issue in of itself, but I wanted to hear your opinion on this overlap.

As for point number 3, I think that's contradicting your proposal at #53, isn't it? Having to backtrack to now say "oops, this item is actually not a macro invocation, my bad" and treating is as text introduces the same problems.

Side note: how is \macro[* hi] supposed to be handled? I assume you'd just treat it as a paragraph, but that then comes with yet another edge case to take care of, so I'd honestly rather be no "markup parameters" at all. Leave that to the block-level syntaxes.

I'm personally pretty on-board with not mixing macros directly inside of paragraphs, and instead giving them their own little region you can execute functions in (doesn't have to be <...>, but it has to be something). The existing macro syntaxes at least are block-level and so do not interfere with anything. In fact, I might do a little deep dive back into my long macros post and revisit the "substitutions" section. If we could find an elegant way in general to do substitution syntax in a paragraph, then that by definition should become the "correct" syntax for us to use.

[boltlessengineer]

Which means there's significant overlap between both syntaxes. It's not much of an issue in of itself, but I wanted to hear your opinion on this overlap.

I don't think they overlap enough to be a problem.

(I will talk in HTML style because I'm a web dev and it's only markup syntax that we all can understand without a problem)

• \macro[markup here] is a macro applied as a span. It's inline, which means it will never include the paragraph break (including zero-width paragraph-break tokens used for some cases). It should result inline elements when expanded to not break the parent paragraph. So we should force this by macro implementation later.
• #macro is a macro applied to a block (which is a paragraph in this case). It consumes a paragraph and can generate a block-type element. I think it is safe to result as non-paragraph elements if we modify an overall syntax a bit more. Like [Suggestion]: Specify that Norg *syntax* isn't a typesetting tool, Neorg is. #35 and [Discussion]: Allow headings anywhere that block-level items can exist #43.

As for point number 3, I think that's contradicting your proposal at #53, isn't it? Having to backtrack to now say "oops, this item is actually not a macro invocation, my bad" and treating is as text introduces the same problems.

Sorry, I should be more verbose about that example.
I mean, both will be two valid paragraphs.

\macro[            < paragraph 1
many paragraphs    < which ends here (due to paragraph break)

contained within   < paragraph 2
]                  < paragraph 2 continues

Paragraph break will have higher precedence than any other inline syntax.
There is some edge-cases (like {*\n\n) but I think it's fine to let them fallback as punctuation just like we did for * followed by whitespace.

Having unbreakable Layer 1 is clear win. I can see many applications easily adopt it without worrying about parsing errors.

Side note: how is \macro[* hi] supposed to be handled? I assume you'd just treat it as a paragraph, but that then comes with yet another edge case to take care of, so I'd honestly rather be no "markup parameters" at all. Leave that to the block-level syntaxes.

Note that I'm a "macro should only expand as an abstract object rather than plain text" kind of person.
All my suggestions are based on this mindset. I still heavily on this side and I haven't heard your opinion enough about this topic. We should really talk about this first.

Edit: typo