Implement JSON grammar with Langium

[bnhmn]

Objective

I want to implement a simple JSON grammar using Langium.
It seems to be working for Xtext with a grammar like this:

grammar org.json.Json with org.eclipse.xtext.common.Terminals

generate json "http://www.json.org/"

Object:
	'{' ((members+=Member) (',' members+=Member)*)? '}';

Member:
	key=STRING ':' value=Value;
	
Value:
	Object | STRING | Array | Boolean | Null | Number;

Array:
	'[' ((values+=Value) (',' values+=Value)*)? ']';

Boolean:
  'true' | 'false';

Null:
  'null';

terminal Number:
	'-'? INT? ('.' INT (('E'|'e') '-'? INT)?)?;

This does not directly work in Langium.
I am getting this error for the terminal rules: This parser rule does not create an object. Add a primitive return type or an action to the start of the rule to force object instantiation.

With some changes it does work:

grammar LangiumJson

entry Objekt:
    "{"
        (properties+=Property)*
    "}";

Property:
    name=STRING ":" value=Value ","?;

Value:
     Objekt | List | Literal;

List:
    "["
        (values+=Value ("," values+=Value)*)?
    "]";

Literal:
    literal=(NULL | BOOL | NUMBER | STRING);


hidden terminal WS: /\s+/;
terminal NULL: "null";
terminal BOOL returns boolean: "true" | "false";
terminal NUMBER returns number: /[+-]?[0-9]+(\.[0-9]*)?/;
terminal STRING: /"(\\.|[^"\\])*"|'(\\.|[^'\\])*'/;

hidden terminal ML_COMMENT: /\/\*[\s\S]*?\*\//;
hidden terminal SL_COMMENT: /\/\/[^\n\r]*/;

While this works fine, I'd prefer the literal type to have the same interface as the "Objekt" type.
Right now I need to access property.value.literal, but instead I want to access it right under property.value, whether it's an object or a literal.

However, replacing the Literal rule with the following does not work:

Literal:
    (NULL | BOOL | NUMBER | STRING)

Questions

1. How can I achieve to assign both objects and primitives to the same "value" property?
2. How can I achieve to convert null tokens into Javascript null or undefined (Langium does not allow a terminal to return null)?
3. How can I convert terminals into custom Object types in general?

Steps To Reproduce

1. Checkout this repository: https://github.com/ben5311/langium-json
2. Run npm install
3. Open repository in VS Code
4. Press F5

[msujew]

Hey @ben5311,

How can I achieve to assign both objects and primitives to the same "value" property?

Currently not possible, though I believe that #947 will alleviate that issue once it gets released in 1.2.0.

How can I achieve to convert null tokens into Javascript null or undefined (Langium does not allow a terminal to return null)?

We didn't encounter that requirement yet, since we usually just use a NullLiteral parser rule for that. Something like

NullLiteral: {infer NullLiteral} 'null';

It didn't make sense to allow null as a possible return value of a terminal in Xtext, since it's indistinguishable from an unparsed expression (which would also return null). With the same line of thinking, we cannot allow undefined as a return type, as it's indistinguishable from an optional AST property, though we could implement null as a possible return type.

How can I convert terminals into custom Object types in general?

Something like this: #758? Currently it's not possible, since a terminal rule needs to return a primitive. We usually follow a relatively simple pattern where we wrap terminals into respective objects. Something like:

StringLiteral: value=STRING;
terminal STRING: ...;