gh-69605: Hardcode some stdlib submodules in PyREPL module completion (os.path, collections.abc...)

[loic-simon]

From #69605 (comment):

This is expected, collections.abc is not a real module, same for os.path so pkgutil won't find it. One thing we could do is hardcode these modules but I haven't had the time to look into it.

This PR adds a hardcoded dictionary of unusual standard library submodules that pkgutil.iter_modules can't find despite them being importable, so we can suggest them anyway.

The hardcoded dictionary comes from checking all documented modules to find which submodules _pyrepl._module_completer.ModuleCompleter was unable to detect.

I wrote a test to ensure all hardcoded modules are indeed importable, but I'm not sure we can programatically detect eventual new modules so we don't forget to add them, but it should be rare enough to not be too much a concern.

cc @tomasr8

• Issue: Readline completion of module names in import statements #69605

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[tomasr8]

Thanks! I was planning to implement this at some point, but you beat me to it :) I haven't checked too deeply, but we should make sure to only autocomplete these submodules when we are sure they are coming from the stdlib. We do not want to autocomplete them when a user intentionally shadows an stdlib module with their own because in that case they might not exist.

[loic-simon]

but we should make sure to only autocomplete these submodules when we are sure they are coming from the stdlib. We do not want to autocomplete them when a user intentionally shadows an stdlib module with their own because in that case they might not exist.

Yeah, I wondered about that, I'll check if we have a reliable and fast way to check this. Just looking at ModuleInfo.file_finder.path to see if it is relative to the stdlib path may do the trick!

---

A side note on this: when testing the current behavior by creating a folder collections with __init__.py and foo.py in it, the current mechanism suggest collections.foo, but is cannot be imported... because collections was already imported at startup (because of interactive mode):

% mkdir collections
% touch collections/__init__.py collections/foo.py
% python
Python 3.14.0rc2 [...]
>>> from collections import <TAB>
>>> from collections import foo
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    from collections import foo
ImportError: cannot import name 'foo' from 'collections' (/Users/loic/cpython/Lib/collections/__init__.py)
>>>
% python -c "from collections import foo; print(foo)"
<module 'collections.foo' from '/Users/loic/test/collections/foo.py'>

Not directly related, but it may be cool to check if the name to complete is already in sys.modules, and if yes only suggest imports from the location of the imported module.

[miss-islington-app]

Thanks @loic-simon for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

[bedevere-app]

GH-138943 is a backport of this pull request to the 3.14 branch.

[loic-simon]

Not directly related, but it may be cool to check if the name to complete is already in sys.modules, and if yes only suggest imports from the location of the imported module.

Adressed in #139461.