Terminal seems to capture more than it should

[altaurog]

I am perplexed. I have the following code:

parser = Lark(
    r"""
    ?start: attrs

    attrs: attr+
    attr: NAME "=" ATTRVALUE

    NAME: /[\w-]+/
    ATTRVALUE: /"(?:[^"]|\\")*(?<!\\)"/
    %import common.WS
    %ignore WS
    """
)

If I apply to the following input:

    subject="Test \"many\" Attributes" author="Aryeh Leib"

I get the following parse tree:

attrs
  attr
    subject
    "Test \"many\" attrs" author="Aryeh Leib"

It seems like ATTRVALUE is matching more than it should. I would expect the result to look like this:

attrs
  attr
    subject
    "Test \"many\" attrs"
  attr
    author
    "Aryeh Leib"

What am I missing?

[MegaIng]

Not enough escapes: \\" gets reduces twice: once by lark to \", and once by the re engine to just ". You can either use four backslashes: \\\\", or (even better) use:

    ATTRVALUE: ESCAPED_STRING
    %import common.ESCAPED_STRING

[MegaIng]

Btw, @erezsh This might also be an interesting change to make for 1.0: Only sequence that get's reduced is \/, everything else is left as is.

[altaurog]

Thank you, it works! (github is busy suggesting that I mark this as an answer, but doesn’t let me mark it as an answer...)

[MegaIng]

No problem. That is one quirk that you are not the first to stumble upon.

[erezsh]

@MegaIng I'm open to the idea. But it needs some serious thought, to make sure it doesn't create any tricky or weird edge cases.