Are macros with "inline matchers" possible?

[Elias-Graf]

Let's suppose I have the following matcher:

pub DefVar: TopLevelExpr = {
    <left: @L> "("  "defvar"  ")" <right: @R> => {
        errors.push(ParseError::ExpectedDefVarName {
            err_span: (left, right).into(),
        });
        TopLevelExpr::Err
    },
    <left: @L> "(" "defvar" Symbol ")" <right: @R> => {
        errors.push(ParseError::ExpectedDefVarValue {
            err_span: (left, right).into(),
        });
        TopLevelExpr::Err
    },
    "(" "defvar" <name:Symbol> <value:Atom> ")" => {
        DefVar::new(name, value).into()
    },
};

Given the nature of the syntax, pretty much every expression is surrounded by parenthesis. With my goal to provide as good of an error message as possible, I'm looking to make the parenthesis "optional", and push an error if that was the case. So I tried creating the following macro:

Paren<Rule>: TopLevelExpr = {
    "(" <Rule> ")" => <>,
    <l:@L> <rule:Rule> ")" <r:@R> => {
        errors.push(ParseError::ExpectedParenthesisOpen {
            err_span: (l, r).into(),
        });
        TopLevelExpr::Err
    },
    <l:@L> "(" <rule:Rule> => {
        errors.push(ParseError::ExpectedParenthesisClose {
            err_span: (l, r).into(),
        });
        TopLevelExpr::Err
    },
};

Now, I would basically like to inline my complete matcher in this macro, but I'm not sure how that would syntactically work, even given just @L, and @R, probably not like this:

pub DefVar: TopLevelExpr = <l:@L> Paren<{
    ... => ...,
    ... => ...,
    ... => ...,
}>  <r:@R>;

Maybe something like this would be possible?

pub DefVar: TopLevelExpr = {
    <left: @L> <Paren<("defvar")>> <right: @R> =>...,
    <left: @L> <Paren<("defvar" Symbol)>> <right: @R> => ...,
    <Paren<("defvar" <name:Symbol> <value:Atom>)>> => ...,
};

What currently seems to work, is creating an "Inner" parser that does not contain any parenthesis, e.g.:

DefVarInner: TopLevelExpr = {
    <left: @L> "defvar" <right: @R> => ...,
    ...
};

and then creating the normal parser like this:

pub DefVar = Paren<DefVarInner>;

But now, left, and right are inside the parenthesis, which isn't terrible, but not what I wanted. In addition, I would have to declare two of every parser, which is a bit "meh" as well.

Is there any clever trick / syntax I'm missing?

Thanks for any help and pointers in advance! :)

[dburgener]

Could you separate the error creation from populating the span?

grammar;

Spanned<Rule> = {
    <left: @L> Paren<Rule> <right: @R> => {
        Rule.span(left, right)
    }
};

Then don't use @L and @R at all in the other rules, and on your return types, define a Span trait with fn span(). Then you can do the "inner" approach you mentioned at the end, but still get the location matching outside the parentheses.

In addition, I would have to declare two of every parser, which is a bit "meh" as well.

I don't follow this part.

An alternative approach, might be to just make all the values options:

grammar;

DefVar: TopLevelExpr = {
    <l: @L> "("? "defvar" <name:Symbol>? <value:Atom>? ")"? <r: @R> = {
        // action code
    }
};

And then check each of the options for None in the action code and return errors accordingly. I haven't thought too hard about whether this approach ends up equivalent after macro expansion or not though. Possibly it would cause parse generation errors.

[dburgener]

So Tok<_> is the type of I, which is the token enum for your custom lexer. The value of I will be whatever you passed to Paren in the places you used the macro. So "defvar" for example.

The action code is the part in the { } brackets after the grammar line. That's the part of the code that you write for what to do when the parser hits a particular rule.

[Elias-Graf]

Yes, I know. Let me try to go top down again. Assuming the following code:

pub DefListenTest: TopLevelExpr = {
    Paren<( "defvar" <name:Symbol> <value:Atom> )> => TopLevelExpr::Err,
};

Paren<I>: TopLevelExpr = {
    "(" <I> ")" => <>,
    <l:@L> <rule:I> ")" <r:@R> => {
        errors.push(ParseError::ExpectedParenthesisOpen {
            err_span: (l, r).into(),
        });
        TopLevelExpr::Err
    },
    <l:@L> "(" <rule:I> <r:@R> => {
        errors.push(ParseError::ExpectedParenthesisClose {
            err_span: (l, r).into(),
        });
        TopLevelExpr::Err
    },
};

I get:

Caused by:
  process didn't exit successfully: `/home/elias/Git/rotten-potato/target/debug/build/rotten-potato-68adc91569ff64c7/build-script-build` (exit status: 101)
  --- stdout
  processing file `/home/elias/Git/rotten-potato/src/grammar.lalrpop`
  /home/elias/Git/rotten-potato/src/grammar.lalrpop:49:11: 49:49 error: named symbols like `name:Symbol` are only allowed at the top-level of a nonterminal

  --- stderr
        Paren<( "defvar" <name:Symbol> <value:Atom> )> => TopLevelExpr::Err,
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  thread 'main' panicked at build.rs:2:57:
  called `Result::unwrap()` on an `Err` value: Kind(InvalidData)
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Removing the named parameters which is of no use to me (I think), since now I can't construct a proper AST-node, but I'll remove them and continue to display further issues:

pub DefListenTest: TopLevelExpr = {
    Paren<( "defvar" <Symbol> <Atom> )> => TopLevelExpr::Err,
};

results in even more issues:

    Blocking waiting for file lock on build directory
   Compiling rotten-potato v0.1.0 (/home/elias/Git/rotten-potato)
error[E0308]: mismatched types
      --> /home/elias/Git/rotten-potato/target/debug/build/rotten-potato-36b8e843c964e903/out/grammar.rs:223630:5
       |
223628 | ) -> TopLevelExpr
       |      ------------ expected `TopLevelExpr` because of return type
223629 | {
223630 |     __0
       |     ^^^ expected `TopLevelExpr`, found `(Symbol, Atom)`
       |
       = note: expected enum `TopLevelExpr`
                 found tuple `(Symbol, Atom)`

For more information about this error, try `rustc --explain E0308`.
error: could not compile `rotten-potato` (lib) due to 1 previous error

The action code does in fact return a TopLevelExpr, in this case, just a hardcoded Err.

[Elias-Graf]

I does not seem to be the result of the action code, but the extracted parameters.

[dburgener]

I does not seem to be the result of the action code, but the extracted parameters.

Yes, that's exactly what I said in my previous reply. The issue with the "(" <I> ")" => <>, part is that you are returning I, which is either a Tok<_>, or a tuple of Toks, depending on the macro call. It's not a TopLevelExpr. You need to build one in the action code for that rule.

However, it does seem like your real problem here is the error about named symbols not at the top level of a non-terminal. That brings me back to this suggestion I made earlier:

Oh yes, I had forgot that aspect. You could still do the same approach though. Just expand your TopLevelExpr::Err type to include the message, and then add the span in and create a ParseError from the top level Spanned macro.

Is there some reason you can't do that in order to avoid passing the tuples into the macro? I don't know of a clean way to get around the top level non-terminal error in this case, and the tuple usage complicates things.

[Elias-Graf]

Yes, that's exactly what I said in my previous reply. The issue with the "(" ")" => <>, part is that you are returning I, which is either a Tok<_>, or a tuple of Toks, depending on the macro call. It's not a TopLevelExpr. You need to build one in the action code for that rule.

Could you show me how that would work? I really don't understand this part. Optimally, I could call the action code defined in DefVarTest / DefListenTest (I've used DefListenTest by mistake earlier, I'll be using DefVarTest for future examples, since it's quite a lot simpler):

pub DefVarTest: TopLevelExpr = {
    Paren<("defvar" <name:Symbol> <value:Atom>)> => {
        DefVar::new(name, value).into()
    },
};

Oh yes, I had forgot that aspect. You could still do the same approach though. Just expand your TopLevelExpr::Err type to include the message, and then add the span in and create a ParseError from the top level Spanned macro.

Yes, I could certainly include the message in the AST, but I fail to see why this would benefit me here. I used TopLevelExpr::Err as a simple placeholder, normally I would use:

pub DefVarTest: TopLevelExpr = {
    Paren<("defvar" <name:Symbol> <value:Atom>)> => {
        DefVar::new(name, value).into()
    },
};

Is there some reason you can't do that in order to avoid passing the tuples into the macro? I don't know of a clean way to get around the top level non-terminal error in this case, and the tuple usage complicates things.

I'm not quite sure what tuple you're talking about? The one resulting from the fact I have two parameters? I mean, I could try and get around that with:

pub DefVarTest: TopLevelExpr = {
    Paren<("defvar" <Symbol> <Atom>)> => {
        let (name, value) = <>;
        DefVar::new(name, value).into()
    },
};

But it really seems like the system is not designed to work in this way. In the Paren macro, I receive the parameters, and now with this code, I'm noticing that I receive the TopLevelExpr, probably from Paren instead:

error[E0308]: mismatched types
      --> /home/elias/Git/rotten-potato/target/debug/build/rotten-potato-36b8e843c964e903/out/grammar.rs:222232:13
       |
222232 |         let (name, value) = __0;
       |             ^^^^^^^^^^^^^   --- this expression has type `TopLevelExpr`
       |             |
       |             expected `TopLevelExpr`, found `(_, _)`
       |
       = note: expected enum `TopLevelExpr`
                 found tuple `(_, _)`