how to capture value in whitespace as side-effects?

[RFoe]

struct scan_macro_directive: lexy::scan_production<void> {
  template <typename Context, typename Reader>
  static constexpr scan_result
  scan(lexy::rule_scanner<Context, Reader> &scanner) {
    scanner.parse(LEXY_LIT("#define"));
    scanner.parse(::lexyd::identifier(::lexyd::ascii::alpha_digit,
                                      ::lexyd::ascii::alpha_digit_underscore));
    return scan_result{true};
  }
};
struct production {
  static constexpr auto whitespace =
      ::lexyd::peek(LEXY_LIT("#")) >> ::lexyd::inline_<scan_macro_directive>;
  static constexpr auto rule = [] consteval noexcept {
    return ::lexyd::terminator(::lexyd::eof)
        .list(::lexyd::identifier(::lexyd::ascii::alpha_digit,
                                  ::lexyd::ascii::alpha_digit_underscore));
  }();
  static constexpr auto value = ::lexy::noop;
};

In template: no type named 'scan_result' in 'lexy::_detail::ws_production<lexyd::_br<lexyd::_peek<lexyd::_lit<char, '#'>, void>, lexyd::_scan>>'clang(typename_nested_not_found)

[foonathan]

By definition, whitespace is unimportant trivia that can be skipped. If you care about it, it should not be using the whitespace rule, but instead you should skip it manually in the appropriate places.

(For the concrete case of preprocessor directives which can literally appear anywhere, I recommend a two-pass approach like the way C/C++ compilers work.)

[RFoe]

Typically in a two-pass approach, the preprocessor pass returns a processed token stream as input for the lexical/syntax pass. However, since Lexy scans the entire string at once, it would still be quite challenging to intercept and process directives with lexy. Or any helpful solutions for that?

[foonathan]

An easy way would be to do one pass that considers lines not starting with # to be whitespace and only parse the directives. Then do a second pass where lines with # are whitespace.

[RFoe]

how does macro expansion work in second pass with lexy? will it be possible to change reader of the statemachine with referenced lexeme_input when meeting preprocessor-token(identifier) and automatically resume to the parent reader after current reader exhausted, like by writing a new rule? if not, i think that will be still tricky to handle pp-directives.

struct replace_preprocessor_token: dsl::rule_base {
    template <typename Reader>
    struct bp
    {
        typename Reader::marker end;
        hash-table-entry                 substring-refered-by-pptoken;
         constexpr bool try_parse(Reader reader)
        {
             // get id by branch_parser_for<identifier>
             // loopup macro hashtable and get substring refered by pptoken(assumed hit)
                  
        }
        template <typename NextParser, typename Context, typename... Args>
        LEXY_PARSER_FUNC auto finish(Context& context, Reader& reader, Args&&... args)
        {
            auto begin = reader.position();

            //context.on(_ev::token{}, lexy::identifier_token_kind, begin, end.position());
            reader.reset(end);

            using continuation = lexy::whitespace_parser<Context, NextParser>;
            return continuation::parse(context, this->substring-refered-by-pptoken, LEXY_FWD(args)...
                            //           ,
                             //          lexy::lexeme<Reader>(begin, end.position())
                            );
        }
    };
};

[foonathan]

Hm, any reaason you can't literally use the C preprocessor to preprocess the input before passing it to lexy?

[RFoe]

That requires to piece together a complete text from the preprocessed token stream(like insight gcc -E directive, perhaps...i'm not sure?), which may seem a little redunctant.