Literal match with capture causing 'invalid input text' error

[BigBallard]

I have a struct that represents a percentage in a particular hex context. The intent here is to match with the starting percent literal but to not capture it, followed by two hex values that would be captured as string.

type Percent struct {
	Value string `"%"@(Hex Hex)`
}

var lexerHexRule = lexer.SimpleRule{Name: "Hex", Pattern: "[0-9]|[A-F]|[a-f]"}

parser, _:= participle.Build[Percent](
	participle.Lexer(lexer.MustSimple([]lexer.SimpleRule{lexerHexRule})),
)

percent, pnErr := parser.ParseString("", "%01")
if pnErr != nil {
        panic(pnErr.Error())
}

pnErr ends up being lexer: invalid input text "%01" How may I alter this setup (without using regex) to work?

[alecthomas]

The lexer needs to be able to match every fragment of valid input text. In your case you do not have a lexer rule for matching %, so you'll need to add one.

[alecthomas]

I really don't know what you're asking. The example works because every token in the grammar is something emitted by the lexer. In this case "scalar" is an Ident. The reason your example is failing is because you are not emitting tokens for all of your input text.

[alecthomas]

Ah maybe this is a misunderstand... Participle is a traditional parser framework, where lexing is a completely separate phase to parsing. So the input text is lexed into tokens that the parser then consumes. If the lexer isn't configured to emit tokens that the parser consumes, parsing will fail because it can't match.

[BigBallard]

I understand what Participle is, but what I am not getting the parser syntax. What I am unable to figure out is how to do something that is similar to the example. What I expect from my case is that the Value string would be %<Hex><Hex>, ie %12 but instead I get an invalid input error. My case is not that much different from the Scalar example above so not sure as to why it is not working.

[BigBallard]

Ok, I think I finally got it figured out. The rules applied to the lexer are not just for capturing. So because I define a lexer rule for a % sign, doesnt mean its just used for capturing, its used for consuming and can also be used for capturing? Probably a bad way of explaining it but thats how my brain is interpreting it.

[alecthomas]

That's correct, any element in the grammar, whether it's capturing or non-capturing, consumes a lexer token.