Skip to content

Commit 4aec8ef

Browse files
Copilottomtau
andcommitted
Add expression parser example demonstrating enums, nested choices, and repetitions
Co-authored-by: tomtau <[email protected]>
1 parent cb4ff43 commit 4aec8ef

File tree

3 files changed

+817
-0
lines changed

3 files changed

+817
-0
lines changed

derive/README.md

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -369,3 +369,145 @@ The `default` attribute generates code that:
369369
3. If parsing fails with other errors, propagates the error
370370

371371
This provides a clean, type-safe way to handle optional grammar elements while keeping your AST representation simple and avoiding the complexity of `Option<T>` handling.
372+
373+
## Advanced Patterns: Enums, Nested Choices, and Repetitions
374+
375+
For more complex grammars involving operators and expression parsing, pest-ast provides patterns for handling:
376+
377+
1. **Parsing into enums** - representing grammar alternatives
378+
2. **Nested choices** - patterns like `(plus | minus)`
379+
3. **Repetitions of anonymous sequences** - patterns like `(operator ~ operand)*`
380+
381+
### Parsing into Enums
382+
383+
When your grammar has alternatives (using `|`), you can use enums to represent them.
384+
385+
**Example Grammar:**
386+
```pest
387+
abc = { a | b | c }
388+
a = { "a" }
389+
b = { "b" }
390+
c = { "c" }
391+
```
392+
393+
**Rust AST:**
394+
```rust
395+
#[derive(FromPest)]
396+
#[pest_ast(rule(Rule::a))]
397+
struct A;
398+
399+
#[derive(FromPest)]
400+
#[pest_ast(rule(Rule::b))]
401+
struct B;
402+
403+
#[derive(FromPest)]
404+
#[pest_ast(rule(Rule::c))]
405+
struct C;
406+
407+
// Enum uses the parent rule that contains the choice
408+
#[derive(FromPest)]
409+
#[pest_ast(rule(Rule::abc))]
410+
enum Abc {
411+
A(A),
412+
B(B),
413+
C(C),
414+
}
415+
```
416+
417+
See `derive/examples/simple_enum_derives.rs` for a complete example.
418+
419+
### Nested Choices with Manual FromPest
420+
421+
For patterns like `arith_expr = { term ~ ((plus | minus) ~ term)* }`, the `(plus | minus)` choice
422+
doesn't have its own grammar rule. You can handle this by implementing `FromPest` manually:
423+
424+
**Example Grammar:**
425+
```pest
426+
plus = { "+" }
427+
minus = { "-" }
428+
arith_expr = { term ~ ((plus | minus) ~ term)* }
429+
```
430+
431+
**Rust AST:**
432+
```rust
433+
#[derive(FromPest)]
434+
#[pest_ast(rule(Rule::plus))]
435+
pub struct Plus;
436+
437+
#[derive(FromPest)]
438+
#[pest_ast(rule(Rule::minus))]
439+
pub struct Minus;
440+
441+
// Manual implementation tries each alternative
442+
#[derive(Debug, Clone, PartialEq)]
443+
pub enum AddOp {
444+
Plus(Plus),
445+
Minus(Minus),
446+
}
447+
448+
impl<'pest> FromPest<'pest> for AddOp {
449+
type Rule = Rule;
450+
type FatalError = from_pest::Void;
451+
452+
fn from_pest(
453+
pest: &mut pest::iterators::Pairs<'pest, Rule>,
454+
) -> Result<Self, from_pest::ConversionError<from_pest::Void>> {
455+
// Try Plus first
456+
if let Ok(plus) = Plus::from_pest(pest) {
457+
return Ok(AddOp::Plus(plus));
458+
}
459+
// Try Minus
460+
if let Ok(minus) = Minus::from_pest(pest) {
461+
return Ok(AddOp::Minus(minus));
462+
}
463+
Err(from_pest::ConversionError::NoMatch)
464+
}
465+
}
466+
```
467+
468+
### Repetitions of Anonymous Sequences
469+
470+
For grammar patterns like `term ~ ((operator ~ operand)*)`, the repeated `(operator ~ operand)` pairs
471+
don't have their own grammar rule. Create a "tail" struct with manual `FromPest` implementation:
472+
473+
**Example Grammar:**
474+
```pest
475+
term = { factor ~ ((mul | div) ~ factor)* }
476+
```
477+
478+
**Rust AST:**
479+
```rust
480+
// One (operator, operand) pair from the repetition
481+
#[derive(Debug)]
482+
pub struct TermTail<'pest> {
483+
pub op: MulOp,
484+
pub factor: Factor<'pest>,
485+
}
486+
487+
impl<'pest> FromPest<'pest> for TermTail<'pest> {
488+
type Rule = Rule;
489+
type FatalError = from_pest::Void;
490+
491+
fn from_pest(
492+
pest: &mut pest::iterators::Pairs<'pest, Rule>,
493+
) -> Result<Self, from_pest::ConversionError<from_pest::Void>> {
494+
// First try to get an operator - if not present, no match
495+
let op = MulOp::from_pest(pest)?;
496+
// Then get the operand
497+
let factor = Factor::from_pest(pest)
498+
.map_err(|_| from_pest::ConversionError::NoMatch)?;
499+
Ok(TermTail { op, factor })
500+
}
501+
}
502+
503+
// The main structure uses Vec<TermTail> for the repetition
504+
#[derive(FromPest, Debug)]
505+
#[pest_ast(rule(Rule::term))]
506+
pub struct Term<'pest> {
507+
pub first: Factor<'pest>,
508+
pub rest: Vec<TermTail<'pest>>, // Handles the (op ~ operand)* part
509+
}
510+
```
511+
512+
For a complete working example demonstrating all these patterns with an expression parser,
513+
see `derive/examples/expression_parser.rs` and `derive/examples/expression_parser.pest`.
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
// Grammar demonstrating patterns for:
2+
// 1. Repetition of anonymous sequences: comparison = { arith_expr ~ (comp_op ~ arith_expr)* }
3+
// 2. Nested choices in repetitions: arith_expr = { term ~ ((plus|minus) ~ term)* }
4+
// 3. Parsing into enums for operators
5+
6+
WHITESPACE = _{ " " | "\t" | "\n" | "\r" }
7+
8+
// Literals - using @ (atomic) to prevent WHITESPACE from being included in spans
9+
number = @{ ASCII_DIGIT+ }
10+
identifier = @{ ASCII_ALPHA ~ (ASCII_ALPHANUMERIC | "_")* }
11+
12+
// Atomic expressions
13+
atom = { number | identifier | "(" ~ expr ~ ")" }
14+
15+
// Multiplicative operators (for nested choice example)
16+
mul = { "*" }
17+
div = { "/" }
18+
19+
// Additive operators (for nested choice example)
20+
plus = { "+" }
21+
minus = { "-" }
22+
23+
// Comparison operators (for nested choice example)
24+
eq = { "==" }
25+
neq = { "!=" }
26+
lt = { "<" }
27+
gt = { ">" }
28+
29+
// Expressions with operator precedence
30+
// Each level demonstrates the pattern: term ~ (operator ~ term)*
31+
32+
// Factor: atom
33+
factor = { atom }
34+
35+
// Term: factor ~ ((mul | div) ~ factor)*
36+
// This demonstrates nested choice (mul | div) in a repetition
37+
term = { factor ~ ((mul | div) ~ factor)* }
38+
39+
// Arithmetic expression: term ~ ((plus | minus) ~ term)*
40+
// This demonstrates nested choice (plus | minus) in a repetition
41+
arith_expr = { term ~ ((plus | minus) ~ term)* }
42+
43+
// Comparison: arith_expr ~ (comp_op ~ arith_expr)*
44+
// This demonstrates using a combined operator rule for cleaner AST
45+
comp_op = { eq | neq | lt | gt }
46+
comparison = { arith_expr ~ (comp_op ~ arith_expr)* }
47+
48+
// Top-level expression
49+
expr = { comparison }

0 commit comments

Comments
 (0)