How do we define expressions that can be used in binary expressions but not at the top level?

[trusktr]

For example, langium-lox has binary expressions. In bytescript (bytescript/bytescript@bddbe1b) we're doing something similar:

https://github.com/bytescript/bytescript/blob/bddbe1b452cbcba256d761a3f391f9ea63c8ebb9/src/language-server/bytescript.langium#L57-L78

// EXPRESSION
Expression infers Expression: // why "infers Expression" here?
	AssignmentExpression;

AssignmentExpression infers Expression:
	SumExpression ({infer BinaryExpression.leftOperand=current} operator="=" rightOperand=SumExpression)*;

SumExpression infers Expression:
	ProductExpression ({infer BinaryExpression.leftOperand=current} operator=("+" | "-") rightOperand=ProductExpression)*;

ProductExpression infers Expression:
	PrimaryExpression ({infer BinaryExpression.leftOperand=current} operator=("*" | "/") rightOperand=PrimaryExpression)*;

PrimaryExpression infers Expression:
	GroupLiteral | NumberLiteral | InvalidParenthesis | PossibleCallExpression;

But as you can see, the function expressions are not included in the Expression chain yet:

https://github.com/bytescript/bytescript/blob/bddbe1b452cbcba256d761a3f391f9ea63c8ebb9/src/language-server/bytescript.langium#L110-L120

FunctionExpression:
	OriginalFunctionExpression | GeneratorFunctionExpression | ArrowFunctionExpression;

OriginalFunctionExpression:
	"function" name=IDENTIFIER? Parameters (":" ReturnType)? body=Block;

GeneratorFunctionExpression:
	"function" "*" name=IDENTIFIER? Parameters (":" ReturnType)? body=Block;

ArrowFunctionExpression:
	Parameters (":" ReturnType)? "=>" body=(ArrowReturnExpression | Block);

and Langium says This rule is declared but never referenced. regarding the FunctionExpressions because they aren't referenced anywhere in the grammar yet.

If I add FunctionExpression to PrimaryExpression, then this causes the function() {}s to be ok at the top level of a file, but nameless functions should not be allowed at the top level of a file (like JavaScript), but it won't even work because it will (I am guessing) conflict with the FunctionDeclaration that is already part of the Statement near the TopLevel:

https://github.com/bytescript/bytescript/blob/bddbe1b452cbcba256d761a3f391f9ea63c8ebb9/src/language-server/bytescript.langium#L15-L22

Statement:
	( VariableDeclaration
	| FunctionDeclaration
	| ReturnStatement
	| Block
	| EmptyStatement
	| Expression
	) ';'?;

So, if I've added it to PrimaryExpression like so:

PrimaryExpression infers Expression:
	GroupLiteral | NumberLiteral | InvalidParenthesis | PossibleCallExpression
	| FunctionExpression;

then this starts to happen:

How can we add function expressions that are not allowed as "statements", while allowing function declarations as statements?

This is on the main branch, at bytescript/bytescript@bddbe1b

[msujew]

If you allow functions at a statement level, then I would recommend to remove them at the top level and add appropriate validations that correctly handle functions at each level. Everything else will probably lead to a lot of headache, at least in my experience.

It's usually for the better to be more flexible during the parsing phase and then prevent invalid programs using the validation feature of Langium. This also allows to create more fine-grained and understandable error messages. Parser errors can be quite cryptic, as you've already pointed out.

[trusktr]

That's what I was starting to think too. Thanks!

There's currently no way to hook in and provide better messages for certain parser errors? Would it even be feasible for that to be a feature?

For now, for example, we wrote this grammar to catch "bad syntax":

https://github.com/bytescript/bytescript/blob/bddbe1b452cbcba256d761a3f391f9ea63c8ebb9/src/language-server/bytescript.langium#L92-L93

InvalidParenthesis:
	empty?="(" ")";

and then we made that an error in the validator:

https://github.com/bytescript/bytescript/blob/bddbe1b452cbcba256d761a3f391f9ea63c8ebb9/src/language-server/bytescript-validator.ts#L216

	checkInvalidParenthesis(expression: InvalidParenthesis, accept: ValidationAcceptor) {
		accept('error', 'SyntaxError: Expected expression.', {node: expression})
	}

It felt less intuitive to make "valid" syntax for invalid syntax so that we can mark it invalid in the validator, but perhaps that's the way.

Will go ahead try loosening the parser and doing the function checking in the validator.

[msujew]

There's currently no way to hook in and provide better messages for certain parser errors? Would it even be feasible for that to be a feature?

Yeah, that should be feasible, but as of now the parser error message generator is hardcoded into Langium. It might make sense to extract it to a service though.

[spoenemann]

👍 Could you create a issue to make it more easily customizable?

[trusktr]

@spoenemann opened issue for customizing parser errors: #1086