Skip to content
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions Doc/library/importlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -985,6 +985,17 @@ find and load modules.
:exc:`ImportError` is raised.


.. class:: NamespacePath(name, path, path_finder)

Represents a namespace package's path.

It uses the module *name* to find its parent module, and from there it looks
up the parent's :attr:`module.__path__`. When this changes, the module's own
path is recomputed, using *path_finder*. The initial value is set to *path*.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we perhaps mention the type of the path finder? (namely using some class Sphinx directive).

Btw, when you say that the initial value is set to path it seems that you are talking about the initial value of the path finder.

In addition, maybe mention that invalidating the caches forces the path to be recomputed (it might not necessarily be the case that the module parent's path has changed).

Finally, could you document the public methods of this new class? For instance there is no insert() method nor remove() as for lists.

I think an example of how to use it could be useful though I don't know whether we want to burden the docs even more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we perhaps mention the type of the path finder? (namely using some class Sphinx directive).

Btw, when you say that the initial value is set to path it seems that you are talking about the initial value of the path finder.

In addition, maybe mention that invalidating the caches forces the path to be recomputed (it might not necessarily be the case that the module parent's path has changed).

I've rewritten the docs to include this.

Finally, could you document the public methods of this new class? For instance there is no insert() method nor remove() as for lists.

Using NamespacePath's mutability at runtime is something that requires folks to be familiar with the source, so I don't think we really need to document that.

The main improvement I want to get from the PR is the ability to isinstance(path, NamespacePath) safely.

I think an example of how to use it could be useful though I don't know whether we want to burden the docs even more.

Similarly, I don't really think that is necessary given the advanced nature of the topic.


For top-level modules, the parent module's path is :data:`sys.path`.


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use .. versionadded:: next + what's new entry? While this only exposes a private class, I think users should be notified more than just with a blurb.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe versionchanged? You can say something like, prior to 3.15, this class was exposed as the module private _NamespacePath class. With 3.15 and beyond, the class has been publicly exposed as NamespacePath.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The attribute is set on importlib._bootstrap_external, which is a private module, so I lean towards versionadded.

.. class:: SourceFileLoader(fullname, path)

A concrete implementation of :class:`importlib.abc.SourceLoader` by
Expand Down
28 changes: 17 additions & 11 deletions Lib/importlib/_bootstrap_external.py
Original file line number Diff line number Diff line change
Expand Up @@ -1079,12 +1079,15 @@ def get_filename(self, fullname):
return self.path


class _NamespacePath:
"""Represents a namespace package's path. It uses the module name
to find its parent module, and from there it looks up the parent's
__path__. When this changes, the module's own path is recomputed,
using path_finder. For top-level modules, the parent module's path
is sys.path."""
class NamespacePath:
"""Represents a namespace package's path.

It uses the module *name* to find its parent module, and from there it looks
up the parent's __path__. When this changes, the module's own path is
recomputed, using *path_finder*. The initial value is set to *path*.

For top-level modules, the parent module's path is sys.path.
"""

# When invalidate_caches() is called, this epoch is incremented
# https://bugs.python.org/issue45703
Expand Down Expand Up @@ -1138,7 +1141,7 @@ def __len__(self):
return len(self._recalculate())

def __repr__(self):
return f'_NamespacePath({self._path!r})'
return f'NamespacePath({self._path!r})'

def __contains__(self, item):
return item in self._recalculate()
Expand All @@ -1147,12 +1150,15 @@ def append(self, item):
self._path.append(item)


_NamespacePath = NamespacePath # for backwards compatibility
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make it a deprecated alias? (and emit a warning if we were to access the private name? that could help transition from the private to the public name).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who's the backwards-compatibility for? People poking at private attributes in private modules?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it hurts, but I also don't think it's necessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well... considering it has been designed to support backwards compatibility, I thought it was worth the deprecation warning so that we can eventually remove it. But that could make the code a bit more complex (as we would need some logic in the module's __getattr__), so I'm also fine with leaving it as is.



# This class is actually exposed publicly in a namespace package's __loader__
# attribute, so it should be available through a non-private name.
# https://github.com/python/cpython/issues/92054
class NamespaceLoader:
def __init__(self, name, path, path_finder):
self._path = _NamespacePath(name, path, path_finder)
self._path = NamespacePath(name, path, path_finder)

def is_package(self, fullname):
return True
Expand Down Expand Up @@ -1207,9 +1213,9 @@ def invalidate_caches():
del sys.path_importer_cache[name]
elif hasattr(finder, 'invalidate_caches'):
finder.invalidate_caches()
# Also invalidate the caches of _NamespacePaths
# Also invalidate the caches of NamespacePaths
# https://bugs.python.org/issue45703
_NamespacePath._epoch += 1
NamespacePath._epoch += 1

from importlib.metadata import MetadataPathFinder
MetadataPathFinder.invalidate_caches()
Expand Down Expand Up @@ -1295,7 +1301,7 @@ def find_spec(cls, fullname, path=None, target=None):
# We found at least one namespace path. Return a spec which
# can create the namespace package.
spec.origin = None
spec.submodule_search_locations = _NamespacePath(fullname, namespace_path, cls._get_spec)
spec.submodule_search_locations = NamespacePath(fullname, namespace_path, cls._get_spec)
return spec
else:
return None
Expand Down
1 change: 1 addition & 0 deletions Lib/importlib/machinery.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from ._bootstrap_external import ExtensionFileLoader
from ._bootstrap_external import AppleFrameworkLoader
from ._bootstrap_external import NamespaceLoader
from ._bootstrap_external import NamespacePath


def all_suffixes():
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Expose :class:`importlib.machinery.NamespacePath`.
Loading