Syntax Highlighting in Github Discussions

[dougransom]

I see some of the masters have syntax highlighting in their discussion posts.

It works like this, using markdown syntax:
```prolog
parent(john, mary).
ancestor(X, Y) :- parent(X, Y).
```

which renders like:

parent(john, mary).
ancestor(X, Y) :- parent(X, Y).

In html, one case use code blocks and highlight.js:
`

<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script> <script>hljs.highlightAll();</script>

parent(john, mary).
ancestor(X, Y) :- parent(X, Y).

`

[bakaq]

The Github Prolog syntax highlighting is very fragile in practice. See for example:

example_term(
   atom1, % Comment
   atom2, % Another comment
   atom3
).

These are valid comments, but the syntax highlighting gets confused. At least it's better than nothing.

For my editors I've had more luck with some tree-sitter based syntax highlighting, but there's the really unfortunate Prolog quirk that it's syntax is stateful because of operators. Tree-sitter grammars build an Concrete Syntax Tree of the code, and they can't be stateful. That means that if you use any operator that is unexpected for the grammar everything breaks down, and there's really no way to get around that. This is really unfortunate, because tree-sitter also allows some structural editing things that would fit Prolog really well.

[mthom]

I've been working on a tree-sitter grammar for ISO Prolog in part to overhaul emacs' prolog-mode to work better with Scryer, Trealla, and other ISO conformant systems. Github uses tree-sitter as you say, so.. could make a contribution there.

[ocharles]

"Proper" Prolog syntax highlighting probably needs to be done over LSP, and can't be done statically in tree-sitter

[mthom]

Also Prolog can change its operators dynamically, unlike Haskell, so that's another source of difficulty not shared by the Haskellers.
I think the parser will need to be extended by a dynamic table of operators somehow. How will I manage that I'm still not sure.

[bakaq]

As far as I know there's no way to parametrize tree-sitter grammars at all, so that's a fundamental limitation. You would probably need to, in the editor or other end user of your grammars, somehow generate and compile and load a grammar with the needed operators on-demand, which sounds really not ideal.

I think that, at least for syntax highlighting, a flat token tree is really the best we can do to have consistent results. That would always identify comments, atoms, variables, strings, graphical char atoms, and other things consistently, and would never break in weird ways because of some unexpected atom being used as an operator (which is the current behavior of all the Prolog grammars that I have tested).

That of course loses the tree structure of the Concrete Syntax Tree and many of the other benefits of Tree-sitter, like structural editing, folding and indenting. Losing structural editing in particular is really unfortunate for Prolog because, just like Lisp, it being homoiconic means that it's perfect for that.

I kinda wish Prolog didn't have custom operators at all, because I've seen the power of Tree-sitter in other languages and I think it would be well worth the trade. ISO demands it though, and many important libraries make use of them, so we can't really work around it.

[triska]

On the other hand, the operators that actually appear in programs tend to be well known in advance. (It's not like operators are randomly generated by programs, and then appear with unforeseen shapes at great surprise to everyone.)

Therefore, it may be feasible to have users generate the grammar they need, based on the operators that they actually use in their programs.

[bakaq]

Therefore, it may be feasible to have users generate the grammar they need, based on the operators that they actually use in their programs.

Yes, that's probably the best way to actually have the benefits of the Tree-sitter CST. It is a bit unwieldy to do in most editors that I know, but with enough plugin shenanigans it should be possible to make a good interface for doing that.