Allow relative imports in the `__main__` module

[FFY00]

Feature or enhancement

Proposal:

Expanding on #109853 (comment)

@zooba, were you envisioning to do this for all scenarios where a __main__ module is exists? I think that makes sense for consistency, but we may want to consider all scenarios to make sure the UX makes sense.

AFAICT, these should be all of them:

(this refers to relative imports in the same directory, it does not cover relative imports to the parent directory)

1. Running a package module via python -m <module>
   • Relative imports already work here, no changes needed.
2. Running a source or extension module via python -m <module>
   • Relative imports don't work here.
   • Allowing relative imports here feels a bit weird, especially if we were to enable -P by default. For it to run, the module would have to be in sys.path, meaning it probably installed rather than being provided by the user, so there isn't much necessity for relative imports.
3. Running a package module directly (python <module.zip>)
   • Relative imports already work here, no changes needed.
4. Running a file directly (python <script.py>)
   • Relative imports don't work here.
5. Running a command string via python -c <command>
   • Relative imports don't work here.
   • There's no canonical base directory to use as a reference, should we use the current directory?
6. Running a command string via the stdin
   • Relative imports don't work here.
   • There's no canonical base directory to use as a reference, should we use the current directory?
7. Running the repl
   • Relative imports don't work here.
   • There's no canonical base directory to use as a reference, should we use the current directory?

I think 4), 5), 6), and 7) are the only use-cases that call for relative imports to work. 2) feels a bit weird, but it may be worth supporting for consistency.

I played a bit with the code, and the implementing this doesn't seem much difficult. I was able to write a working PoC targeting only 4) with much difficulty, so I don't think that's a worry.

Considering this, do we think it makes sense to make this change?
If so, should it cover all scenarios, or do we want to keep some of them as-is?

For further reflection, how much of an actual improvement would this actually be over the current behavior of adding the current directory to sys.path?
I can't help but feel a bit like we are trading one weird behavior for another.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

[zooba]

The -P option complicates things, but making from . import spam work as "import spam but only from sys.path[0] when in __main__" seems like a reasonable (brief) specification.

In practice, we'd probably prefer to set __main__.__path__ rather than adding tons of new logic, and likely -P would have to suppress it in some cases (e.g. where CWD is the only option we've got).

The main thing I'd say is that if it's not simple to do, or its behaviour can't be reasonably inferred from existing behaviours, we probably shouldn't do it.

[FFY00]

In practice, we'd probably prefer to set __main__.__path__ rather than adding tons of new logic, and likely -P would have to suppress it in some cases (e.g. where CWD is the only option we've got).

That's what I did in my PoC. It is enough to get importlib to behave on relative imports.

In which scenarios would -P suppress this behavior? This working with -P was the main point of the proposal.

[zooba]

I think the scenarios where you said "There's no canonical base directory to use as a reference" shouldn't enable relative imports with -P. Essentially, anywhere that sys.path[0] without -P would be CWD only because we don't have a better choice.