Skip to content

Commit 739f9d7

Browse files
committed
Update for changes to macro parser
1 parent c221508 commit 739f9d7

File tree

1 file changed

+21
-35
lines changed

1 file changed

+21
-35
lines changed

src/macro-expansion.md

Lines changed: 21 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -517,8 +517,9 @@ We use these items in macro parser:
517517
are about to ask the MBE parser to parse. We will consume the raw stream of
518518
tokens and output a binding of metavariables to corresponding token trees.
519519
The parsing session can be used to report parser errors.
520-
- a `matcher` variable is a sequence of [`MatcherLoc`]s that we want to match
521-
the token stream against. They're converted from token trees before matching.
520+
- a `matcher` variable is a sequence of [`MatcherLoc`]s that we want to match the token stream
521+
against. They're converted from the original token trees in the macro's definition before
522+
matching.
522523

523524
[`MatcherLoc`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_expand/mbe/macro_parser/enum.MatcherLoc.html
524525

@@ -544,41 +545,26 @@ The full interface is defined [here][code_parse_int].
544545
The macro parser does pretty much exactly the same as a normal regex parser
545546
with one exception: in order to parse different types of metavariables, such as
546547
`ident`, `block`, `expr`, etc., the macro parser must call back to the normal
547-
Rust parser. Both the definition and invocation of macros are parsed using
548-
the parser in a process which is non-intuitively self-referential.
549-
550-
The code to parse macro _definitions_ is in
551-
[`compiler/rustc_expand/src/mbe/macro_rules.rs`][code_mr]. It defines the
552-
pattern for matching a macro definition as `$( $lhs:tt => $rhs:tt );+`. In
553-
other words, a `macro_rules` definition should have in its body at least one
554-
occurrence of a token tree followed by `=>` followed by another token tree.
555-
When the compiler comes to a `macro_rules` definition, it uses this pattern to
556-
match the two token trees per the rules of the definition of the macro, _thereby
557-
utilizing the macro parser itself_. In our example definition, the
558-
metavariable `$lhs` would match the patterns of both arms: `(print
559-
$mvar:ident)` and `(print twice $mvar:ident)`. And `$rhs` would match the
560-
bodies of both arms: `{ println!("{}", $mvar); }` and `{ println!("{}", $mvar);
561-
println!("{}", $mvar); }`. The parser keeps this knowledge around for when it
562-
needs to expand a macro invocation.
563-
564-
When the compiler comes to a macro invocation, it parses that invocation using
565-
a NFA-based macro parser described above. However, the matcher variable
566-
used is the first token tree (`$lhs`) extracted from the arms of the macro
567-
_definition_. Using our example, we would try to match the token stream `print
568-
foo` from the invocation against the matchers `print $mvar:ident` and `print
569-
twice $mvar:ident` that we previously extracted from the definition. The
570-
algorithm is exactly the same, but when the macro parser comes to a place in the
571-
current matcher where it needs to match a _non-terminal_ (e.g. `$mvar:ident`),
572-
it calls back to the normal Rust parser to get the contents of that
573-
non-terminal. In this case, the Rust parser would look for an `ident` token,
574-
which it finds (`foo`) and returns to the macro parser. Then, the macro parser
575-
proceeds in parsing as normal. Also, note that exactly one of the matchers from
576-
the various arms should match the invocation; if there is more than one match,
577-
the parse is ambiguous, while if there are no matches at all, there is a syntax
548+
Rust parser.
549+
550+
The code to parse macro definitions is in [`compiler/rustc_expand/src/mbe/macro_rules.rs`][code_mr].
551+
For more information about the macro parser's implementation, see the comments in
552+
[`compiler/rustc_expand/src/mbe/macro_parser.rs`][code_mp].
553+
554+
Using our example, we would try to match the token stream `print foo` from the invocation against
555+
the matchers `print $mvar:ident` and `print twice $mvar:ident` that we previously extracted from the
556+
rules in the macro definition. When the macro parser comes to a place in the current matcher where
557+
it needs to match a _non-terminal_ (e.g. `$mvar:ident`), it calls back to the normal Rust parser to
558+
get the contents of that non-terminal. In this case, the Rust parser would look for an `ident`
559+
token, which it finds (`foo`) and returns to the macro parser. Then, the macro parser continues
560+
parsing.
561+
562+
Note that exactly one of the matchers from the various rules should match the invocation; if there is
563+
more than one match, the parse is ambiguous, while if there are no matches at all, there is a syntax
578564
error.
579565

580-
For more information about the macro parser's implementation, see the comments
581-
in [`compiler/rustc_expand/src/mbe/macro_parser.rs`][code_mp].
566+
Assuming exactly one rule matches, macro expansion will then *transcribe* the right-hand side of the
567+
rule, substituting the values of any matches it captured when matching against the left-hand side.
582568

583569
## Procedural Macros
584570

0 commit comments

Comments
 (0)