Skip to content

How to change or add a new custom parser error message

Jump to bottom
rybern edited this page Mar 11, 2020 · 4 revisions

Background

Stanc3 uses Menhir's Incremental API to provide custom error messages for when the parser detects an error. Menhir and the Incremental API are documented in the reference manual. We write the custom error messages in src/frontend/parser.messages file, which follows Menhir's messages format (documented here). Those messages are automatically integrated into the parser by the build rules defined in src/frontend/dune.

Each rule in parser.messages indicates the error state it corresponds to by specifying any stack of tokens which result in that error state. There may be many such stacks of tokens which could correspond to the same error, and Menhir will point out if there are two rules defined for the same error.

Typical process

  1. Find the existing error message for the parser error in question by either:
    • Searching the message file for the current error message test,
    • Compiling a stan program which throws the error message with the --debug-parse flag, then search for the error state number
    • Add a new rule to parser.messages corresponding to any stack of tokens which could trigger the error you're interested in, then let Menhir tell you the line number of the existing rule the new rule collides with
  2. Update the message of the existing rule

Example

Suppose I want to change the error message for when the program is missing a '{' after the 'model' keyword.

I write an example Stan program that has the error:

parameters {
  real x;
}
model
  x ~ normal(0, 1);
}

Now I compile this program with stanc3 using the --debug-parse option. The result:

...
Expected "{" after "model".
(Parse error state 685)

The parse error state I'm looking for is 685. I search the parser.messages file for "685" (perhaps with the command grep -n 685 src/frontend/parser.messages), and I find:

program: MODELBLOCK WHILE
##
## Ends in an error in state: 685.
##
## model_block -> MODELBLOCK . LBRACE list(vardecl_or_statement) RBRACE [ GENERATEDQUANTITIESBLOCK EOF ]
##
## The known suffix of the stack is as follows:
## MODELBLOCK
##

Expected "{" after "model".

I can now replace this message with my new version, and remove my redundant `MODELBLOCK REAL` rule.

Adding new parser states

Suppose I actually wanted to add a new message that would only show when a more specific error is made; maybe I want a special message for when `model` followed by a declaration. To do this I need to change the parse grammar in src/frontent/parser.mly to split off a new error state for the situation I want to catch. However, I need to make sure not to change the behavior of the parser, so I should guarantee that he new rule will never successfully be built by terminating it with a token that can never occur. For example, I could change `model = MODELBLOCK LBRACE …` to `model = MODELBLOCK LBRACE … | MODELBLOCK REAL UNREACHABLE`. This is enough to create a new parser state with a different error message.

After creating a new parser state, you should run dune build @update_messsages in the src/frontend directory to check the new entry and generate the state comment (like the parser error number).

Clone this wiki locally