ast.parse(mode='single') exception for block statements without trailing newline

[tom-pytel]

Bug report

Bug description:

I deliberated on whether to post this as its so inconsequential, but the fix (in ast.py at least, dunno about compile) is super trivial so might as well. It goes back to 3.10 at least.

>>> ast.parse('pass', mode='exec')
Module(body=[Pass()], type_ignores=[])

>>> ast.parse('pass', mode='single')
Interactive(body=[Pass()])

>>> ast.parse('if 1: pass', mode='exec')
Module(body=[If(test=Constant(value=1, kind=None), body=[Pass()], orelse=[])], type_ignores=[])

>>> ast.parse('if 1: pass', mode='single')
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    ast.parse('if 1: pass', mode='single')
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tom/work/cpython/main/cp/Lib/ast.py", line 46, in parse
    return compile(source, filename, mode, flags,
                   _feature_version=feature_version, optimize=optimize)
  File "<unknown>", line 1
    if 1: pass
SyntaxError: invalid syntax

>>> ast.parse('if 1: pass\n', mode='single')
Interactive(body=[If(test=Constant(value=1, kind=None), body=[Pass()], orelse=[])])

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

[brianschubert]

This appears to be consistent with the documented behavior. When mode="single", the docs say that source is parsed as an interactive input, which requires a trailing newline:

Input in interactive mode is parsed using the following grammar:

interactive_input ::= [stmt_list] NEWLINE | compound_stmt NEWLINE

Note that a (top-level) compound statement must be followed by a blank line in interactive mode; this is needed to help the parser detect the end of the input.

[tom-pytel]

This appears to be consistent with the documented behavior. When mode="single", the docs say that source is parsed as an interactive input, which requires a trailing newline:

A little counterintuitive that pass and x = 1; y = 2 parse without newline, but if its in the docs... What say you @picnixz, close it?

[picnixz]

Well, strictly speaking pass is not a compound statement but a simple one, but ast.parse('pass', mode='single') works. So I'm not really sure about that. I'm fine with leaving as is or rejecting ast.parse('pass', mode="single") (but I wouldn't want that honestly if it's only for consistency's sake).

WDYT @JelleZijlstra

[JelleZijlstra]

No opinion here, feels like more of a question for the parser maintainers.

[picnixz]

cc @pablogsal

[pablogsal]

As surprising as it is, this is the correct behavior. Without the NEWLINE the parser doesn't know the compound has finished. This was the same with the old parser as well:

>>> import ast
>>> ast.parse('if 1: pass', mode='single')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pablogsal/github/python/3.10/Lib/ast.py", line 47, in parse
    return compile(source, filename, mode, flags,
  File "<unknown>", line 1
    if 1: pass
             ^
SyntaxError: unexpected EOF while parsing

I agree is kind of surprising but even if we wanted to change it (which I would say we don't) we would need to deprecate it and then change it and that would not only take many releases but I can promise you it will break people that are relying on the failure to detect incomplete input.

So unfortunately I think we need to close this. Thanks for opening the issue though.

CC @lysnikolaou for visibility in case he has a different view

[lysnikolaou]

FWIW I agree with what @pablogsal said. Also, I think that both the new REPL and the IDLE REPL would break if we were to change this, which we would then need to circumvent.

[tom-pytel]

No worries, its not much of an "issue" to begin with, more of a quirk that's easy enough to code around.