Inquiring about the function of `Parser->level` in `parser.c`, maybe it can be deleted when enable-optimization?

[dongrixinyu]

Feature or enhancement

Proposal:

I am reviewing the cpython/Parser/parser.c and notice many p->level operations in codes like below:

...
         {
            D(fprintf(stderr, "%*c+ term[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "term '//' factor"));
            Token *_token = _PyPegen_get_last_nonnwhitespace_token(p);
            if (_token == NULL) {
                p->level--;
                return NULL;
            }
            int _end_lineno = _token->end_lineno;
            UNUSED(_end_lineno); // Only used by EXTRA macro
            int _end_col_offset = _token->end_col_offset;
            UNUSED(_end_col_offset); // Only used by EXTRA macro
            _res = _PyAST_BinOp ( a , FloorDiv , b , EXTRA );
            if (_res == NULL && PyErr_Occurred()) {
                p->error_indicator = 1;
                p->level--;
                return NULL;
            }
            goto done;
        }
done:
    p->level--;  #  Here is the line
    return _res;

It seems that the level attribute of struct Parser only works when debug fprintf like D(fprintf(stderr, "%*c+ term[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "term '//' factor"));

According to this, I think p->level++ and p->level-- can be wrapped by D(*) for optimization.

But I still have no idea if this operation may cause other corruption. I will pull request if what I consider is totally correct.

Besides, this code occured in most functions. set it in a #define may be more suitable.

    if (p->level++ == MAXSTACK) {
        _Pypegen_stack_overflow(p);
    }

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

[rruuaanng]

The D macro is for controlling error messages, not for general use.

[rruuaanng]

And to keep the parser easy to read, avoid putting multiple statements in a macro.

[picnixz]

Increment and decrement of levels do not depend on whether you use them for debugging or not.

It seems that the level attribute of struct Parser only works when debug fprintf like D(fprintf(stderr, "%*c+ term[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "term '//' factor"));

No, p->level is also used by

    if (p->level++ == MAXSTACK) {
        _Pypegen_stack_overflow(p);
    }

So you definitely need to keep those calls. By the way, this file is generated by Tools/peg_generator/parser/c_generator.py.

[dongrixinyu]

Thx, I got your point.