Replace combined injections with "scoped" injections

[the-mikedavis]

Context about combined injections...

Injections are a tree-sitter feature that lets you treat some part of a document as another language. That "injected" range is parsed with that injected language's parser instead of the parser for the document's language and you can run queries across these injections to get highlighting of other languages. For example highlighting in a code fence in markdown is done with a query like this:

(fenced_code_block
  (info_string (language) @injection.language)
  (code_fence_content) @injection.content)

Everything in the code_fence_content node is captured as the injection.content and we use the parser for the language captured by injection.language for that range. You might also highlight HTML within markdown like this:

((html_block) @injection.content
 (#set! injection.language "html")
 (#set! injection.combined))

so you can get HTML highlighting on something like <code>"Hello, world!"</code>.

Combined injections let you piece together multiple nodes captured by injection.content and treat them as one document. For example

<details>

This is more markdown...

</details>

The <details> and </details> parts are captured as injection.content and since this pattern sets injection.combined, what tree-sitter-html's parser and queries see is this document: <details></details>. That's good for tree-sitter-html since it detects open/close tag pairs so it needs to see both nodes in the same document. Without injection.combined, tree-sitter-html would essentially see one document with <details> and another document with </details> and the highlighting would be incorrect.

---

Combined injections are very useful for languages like markdown but they have a rough edge when it comes to other languages: everything tagged with injection.content that sets injection.combined all becomes one document. It would be useful for some languages to have more granularity.

For example you might have a templated language that has nodes like (start_tag), (middle_tag) and (end_tag) and you want to inject each of those tags with some other language, but you don't want the injection to be "global" - you want the injected parser to see a document just for this match.

((start_tag) @injection.content
 (middle_tag) @injection.content
 (end_tag) @injection.content
 (#set! injection.language "somelang")
 (#set! injection.scope match))

This would mean that the injected language's parser would see a document just consisting of the contents of (start_tag), (middle_tag) and (end_tag). If you have a document with multiple blocks of start+middle+end, each block is injected separately.

What we call combined injections today would be a global scope:

((html_block) @injection.content
 (#set! injection.language "html")
 (#set! injection.scope global))

The current behavior of injection.content when injection.combined is not set would be (#set injection.scope capture). So in summary, three possible scopes:

• global: what is injection.combined today - all ranges for a language are treated as one injection
• match: not possible today - all injection.content belonging to an individual match are treated as one injection
• capture: what is injection.content today without injection.combined - the capture is the only range in the injection

The motivation for this is EEx and similar templating languages. Elixir's parser is somewhat sensitive to whitespace so when two expressions are next to each other without a newline in between they are parsed differently than you would expect. Elixir's parser works well normally, just not well when injected in EEx. We have start and middle/end nodes available in EEx that can be matched on for one of these match scoped injections so that we don't give tree-sitter-elixir more than one expression per injection. This isn't an issue in EJS for example because the JavaScript parser is resilient enough to whitespace that it can handle a global document with multiple expressions next to each other.

[pascalkuthe]

I had an idea for defining scope that came to mine while reading this.

Instead of having set! injection.scope global/match/capture we could allow an arbitrary scope name and all captures with the same name endup being combined. If no scope is set it will always be a non-combined injection. Global would be achieved by setting a commonstring for all injections (likely just "global" but it doesn't really matter).

The match case probably still makes sense on second taught since you can't quite cover that with this. Named scopes may still be useful tough but probably more orthogonal than I first taught

[lewis6991]

You still need to differentiate whether an injection scope applies to one match or all matches of a pattern. I assume match scope would match to one instance, whereas global would apply to all matches of the pattern.