Unreliable errors

[vasilache-radu]

Dear community, I have noticed many times inconsistencies between the errors, code suggestions, autocomplete and the grammar. Now I encounter an error in the extension development host regarding the parsing of a float value. It used to work before, now it doesn't, therefore I call it inconsistency. I tried the same grammar in the playground and there is no error. Why is this happening? Is there a local problem on my machine?

[msujew]

Hey @vasilache-radu,

do you mind sharing a playground link? Images are pretty unhelpful when it comes to debugging issues.

Note that the Playground doesn't use the current version of Langium (3.4, whereas the playground uses 3.0). There might be slight differences in behavior.

[vasilache-radu]

Sure, here is the link.

[msujew]

I've just tried setting up a minimal example using Langium 3.4 and yo, but everything is working as expected in VS Code:

Which version of Langium are you using in your project?

[vasilache-radu]

Interesting. It's true I haven't updated it since last year. Could you please guide me how to check the version and eventually update it?

[msujew]

You can use npm list langium to get the current version of Langium installed in your project. To update it, pull up all versions of langium and langium-cli in your package.json to the same version. See langium on npm to see which versions are available.

[vasilache-radu]

Hei @msujew

Thank you for your help. I think the problem was indeed the version.
After updating I had a lot of errors and I decided to create a new project and copy the files that I wrote in the previous one. Is there an easier way to update the project and solve dependencies or other problems automatically?

I would appreciate if you could also check the following grammar that I wrote in the playground. There it works as expected, but on my machine, the values 0, 1,... are not seen as INTEGER, which I guess is a parsing problem. Now my machine has Langium 3.4.0, do you encounter the same problem with the grammar?

[msujew]

Is there an easier way to update the project and solve dependencies or other problems automatically?

Updating from one framework version to another (with a bunch of breaking changes in between) is simply an involved process. Unless AI becomes sophisticated enough to solve that on its own, there's no way to automate this.

Now my machine has Langium 3.4.0, do you encounter the same problem with the grammar?

I cannot reproduce. The grammar you've given does work in my 3.4.0 project. Note that Langium's parser doesn't work like Xtext's - for example, the ordering of terminals is important. The tokenizer uses first-fit instead of longest-fit to match terminals.

[vasilache-radu]

Hi @msujew,

Thank you for your help and sorry for the late reply.
Regarding the grammar I asked you to check, as I gave it to you it also works for me. It was part of a bigger grammar which I have not shared and it didn't work. I have figured the problem out now, and the issue I discovered was new to me so I will share it here.

Check out this grammar. In MappingValue, value=INTEGER is not parsed correctly because terminal rule INTEGER comes after terminal rule FLOAT. I knew the Chevrotain parser behaves like this. But what I find interesting is that terminal rule FLOAT is treated as it doesn't exist when there is no explicit rule-chain that calls it. So the parser tries to match the token with the first matchy rule, but it considers only the rules that are explicitly mentioned on one of the rules branches.

[msujew]

But what I find interesting is that terminal rule FLOAT is treated as it doesn't exist when there is no explicit rule-chain that calls it.

Yes, we only create tokens for rules that are reachable by the parser. Otherwise, there's little use for those tokens (except for generating parser errors whenever they appear). See also:

langium/packages/langium/src/parser/token-builder.ts

Line 52 in 89c0a9d

const reachableRules = stream(getAllReachableRules(grammar, false));