support lua5.5 grammar

[CppCXY]

No description provided.

[clason]

You really need to use the latest version of the tree-sitter-cli (0.26.3) if you want to contribute to grammars. (In this case, you can fake it by just manually editing the modeline, but that just adds unnecessary friction in the long run.)

[clason]

Parse failures look to be genuine regressions:
https://github.com/xmake-io/xmake/blob/7ee27a5e5a58b4d8a51ec0d2c0fc153239cb6722/xmake/actions/global/main.lua#L41
https://github.com/xmake-io/xmake/blob/7ee27a5e5a58b4d8a51ec0d2c0fc153239cb6722/xmake/actions/global/menuconf.lua#L289
https://github.com/xmake-io/xmake/blob/7ee27a5e5a58b4d8a51ec0d2c0fc153239cb6722/xmake/core/base/global.lua#L66

Looks like you can't make global a reserved function without breaking valid Lua 5.1-5.4 code.

@MunifTanjim I think this requires deeper thought about how this grammar handles the various (deliberately incompatible) dialects of Lua. One option would be generate-time extensions like https://github.com/tree-sitter-grammars/tree-sitter-markdown. (I feel like we've been down that hole before...)

[CppCXY]

In a handwritten parser we can first try treating global as a regular identifier, parse one or two tokens using Lua 5.5 rules, and that will be enough to determine whether it matches the corresponding syntax.
but I don't know whether tree-sitter supports doing this.

[CppCXY]

Similarly, goto is another example — LuaJIT supports using goto as a field.

LuaJIT 2.1.0-beta3 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/
JIT: ON SSE2 SSE3 SSE4.1 BMI2 fold cse dce fwd dse narrow loop abc sink fuse
> l = {} l.goto = {}
> print(l.goto)
table: 0x3dcf22c0

[clason]

In a handwritten parser we can first try treating global as a regular identifier, parse one or two tokens using Lua 5.5 rules, and that will be enough to determine whether it matches the corresponding syntax.
but I don't know whether tree-sitter supports doing this.

It does not, at least not without significantly (and unacceptably) complicating the external scanner.

Similarly, goto is another example — LuaJIT supports using goto as a field.

LuaJIT is Lua 5.1, where goto is not a reserved keyword. So goto itself is not treated specially in the grammar; instead there is a dedicated rule for "goto statements".

[CppCXY]

I think we can do workaround: replace global with token(seq('global', /\s/))

[clason]

I don't think that's a good idea... This will include surrounding whitespace in the corresponding node, which is an absolute deal-breaker for some use cases (which go far beyond highlighting).

[MunifTanjim]

Well... This PR is still not completely anyway... Apparently this is valid in Lua 5.5

🤷🏼‍♂️

global is keyword, yeah... but it can also be a variable 🤣

[clason]

Lua 5.5 was a mistake...

Joking aside, would a dedicated global_statement rule analogously to goto_statement work here?

[MunifTanjim]

Looks like other languages have this issue too, e.g. tree-sitter-c-sharp

[MunifTanjim]

@CppCXY can you try this?

diff --git a/grammar.js b/grammar.js
index 08a1373..851cbf2 100644
--- a/grammar.js
+++ b/grammar.js
@@ -486,7 +486,7 @@ export default grammar({
 
     // var ::=  Name | prefixexp [ exp ] | prefixexp . Name
     variable: ($) =>
-      choice($.identifier, $.bracket_index_expression, $.dot_index_expression),
+      choice($._contextual_keyword, $.identifier, $.bracket_index_expression, $.dot_index_expression),
     // prefixexp [ exp ]
     bracket_index_expression: ($) =>
       seq(
@@ -627,5 +627,8 @@ export default grammar({
           field('end', alias($._block_comment_end, ']]'))
         )
       ),
+
+    // only `global` for now
+    _contextual_keyword: (_) => 'global',
   },
 });

[CppCXY]

can not pass like:

{
    global = 1
}

[MunifTanjim]

can not pass like:

{
    global = 1
}

Try this?

diff --git a/grammar.js b/grammar.js
index 851cbf2..f8f820a 100644
--- a/grammar.js
+++ b/grammar.js
@@ -543,7 +543,7 @@ export default grammar({
           '=',
           field('value', $.expression)
         ),
-        seq(field('name', $.identifier), '=', field('value', $.expression)),
+        seq(field('name', choice($._contextual_keyword, $.identifier)), '=', field('value', $.expression)),
         field('value', $.expression)
       ),
 

[CppCXY]

now, all ci test passed

[MunifTanjim]

LGTM 👍🏼