Improved grammar checking

[ericvergnaud]

@KvanTTT I believe you have lots of ideas on this topic ?

[KvanTTT]

At first we can try to aggregate not implemented open issues from the original ANTLR repository.

[KvanTTT]

I'm wondering if it's possible to transfer issues from ANTLR 4 to ANTLR 5 repository? I don't have such an option but maybe you have, since you are admin? It would be very useful.

I've checked my own repositories and it works:

[ericvergnaud]

I can try if you tell me which issues you’d like to transfer ?

…

Le 27 janv. 2024 à 23:01, Ivan Kochurkin ***@***.***> a écrit : I'm wondering if it's possible to transfer issues from ANTLR 4 to ANTLR 5 repository? I don't have such an option but maybe you have, since you are admin? It would be very useful. I've checked my own repositories and it works: image.png (view on web) <https://github.com/antlr/antlr5-specs/assets/1150330/a0ad616f-8abf-402a-a63e-e691de5d0eac> — Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZNQJCZWHL7ZV7DRACCJVLYQV2KXAVCNFSM6AAAAABAW6ZFISVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DENRXG44TM>. You are receiving this because you authored the thread.

[KvanTTT]

Let's begin with https://github.com/antlr/antlr4/issues/3640 - I have a fix for that.

[KvanTTT]

@ericvergnaud can you try to move it? I'm wondering if it works fine and I want to add info about the new issue in one of my commit messages.

[kaby76]

I'm still developing a list of checks, but I just came across this one this evening while investigating another antipattern.

exp : 
   ......
   | exp exp
   | ...
   ;

Concatenation is usually specified this way in papers but it really is an "Antlr antipattern" because it is ambiguous. I'm still trying to write the XPath expression detect this. And, although it is not correct (//alternative/element[./atom//(TOKEN_REF | RULE_REF)/text() = following-sibling::*[1]/atom//(TOKEN_REF | RULE_REF)/text() and ancestor::parserRuleSpec/RULE_REF/text() = ./atom//(TOKEN_REF | RULE_REF)/text()]/atom//(TOKEN_REF | RULE_REF)/text()), we see this in a few grammars in gramamrs-v4 (e.g., awk, nanofuck, hypertalk, z).

[KvanTTT]

Have you investigated how ANTLR unfold such kind of left recursion? (ANTLR internally converts left-recursive expression into non-left-recursive ones, see LeftRecursiveRuleTransformer.java).

[kaby76]

You're right--there may not be ambiguity nor fallbacks. I was investigating patterns with kleene operators or recursions, e.g., e : a a ; a : 'a'+;, or e : a a ; a : 'a' a | 'a' ;, or 'e : e e | a ; a : 'a'+;, or 'e : ('a'+) * ;, all of which cause ambiguity and fallbacks. I thought there was already a check in the tool for closures of closures, but the last example isn't flagged. Perhaps I was thinking of a closure of something that derives empty, e.g., e : ('a'*) +;, which is flagged by the tool.