Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 98 additions & 4 deletions Doc/library/annotationlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -368,6 +368,10 @@ Functions
doesn't have its own annotations dict, returns an empty dict.
* All accesses to object members and dict values are done
using ``getattr()`` and ``dict.get()`` for safety.
* Supports objects that provide their own :attr:`~object.__annotate__` method,
such as :class:`functools.partial` and :class:`functools.partialmethod`.
See :ref:`below <functools-objects-annotations>` for details on using
:func:`!get_annotations` with :mod:`functools` objects.

*eval_str* controls whether or not values of type :class:`!str` are
replaced with the result of calling :func:`eval` on those values:
Expand All @@ -389,10 +393,12 @@ Functions
``sys.modules[obj.__module__].__dict__`` and *locals* defaults
to the *obj* class namespace.
* If *obj* is a callable, *globals* defaults to
:attr:`obj.__globals__ <function.__globals__>`,
although if *obj* is a wrapped function (using
:func:`functools.update_wrapper`) or a :class:`functools.partial` object,
it is unwrapped until a non-wrapped function is found.
:attr:`obj.__globals__ <function.__globals__>`.
If *obj* has a ``__wrapped__`` attribute (such as functions
decorated with :func:`functools.update_wrapper`), or if it is a
:class:`functools.partial` object, it is unwrapped by following the
``__wrapped__`` attribute or :attr:`~functools.partial.func` attribute
repeatedly to find the underlying wrapped function's globals.

Calling :func:`!get_annotations` is best practice for accessing the
annotations dict of any object. See :ref:`annotations-howto` for
Expand Down Expand Up @@ -421,6 +427,94 @@ Functions

.. versionadded:: 3.14

.. _functools-objects-annotations:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this doesn't belong in the annotationlib docs, maybe in the functools ones.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a different idea here. plz correct me if I'm wrong

It seems that we recommend people use annotationlib to process the annotation here. I prefer add a more detail document in here and reference this in functools


Using :func:`!get_annotations` with :mod:`functools` objects
--------------------------------------------------------------

:func:`get_annotations` has special support for :class:`functools.partial`
and :class:`functools.partialmethod` objects. When called on these objects,
it returns only the annotations for parameters that have not been bound by
the partial application, along with the return annotation if present.

For :class:`functools.partial` objects, positional arguments bind to parameters
in order, and the annotations for those parameters are excluded from the result:

.. doctest::

>>> from functools import partial
>>> def func(a: int, b: str, c: float) -> bool:
... return True
>>> partial_func = partial(func, 1) # Binds 'a'
>>> get_annotations(partial_func)
{'b': <class 'str'>, 'c': <class 'float'>, 'return': <class 'bool'>}

Keyword arguments in :class:`functools.partial` set default values but do not
remove parameters from the signature, so their annotations are retained:

.. doctest::

>>> partial_func_kw = partial(func, b="hello") # Sets default for 'b'
>>> get_annotations(partial_func_kw)
{'a': <class 'int'>, 'b': <class 'str'>, 'c': <class 'float'>, 'return': <class 'bool'>}

For :class:`functools.partialmethod` objects accessed through a class (unbound),
the first parameter (usually ``self`` or ``cls``) is preserved, and subsequent
parameters are handled similarly to :class:`functools.partial`:

.. doctest::

>>> from functools import partialmethod
>>> class MyClass:
... def method(self, a: int, b: str) -> bool:
... return True
... partial_method = partialmethod(method, 1) # Binds 'a'
>>> get_annotations(MyClass.partial_method)
{'b': <class 'str'>, 'return': <class 'bool'>}

When a :class:`functools.partialmethod` is accessed through an instance (bound),
it becomes a :class:`functools.partial` object and is handled accordingly:

.. doctest::

>>> obj = MyClass()
>>> get_annotations(obj.partial_method) # Same as above, 'self' is also bound
{'b': <class 'str'>, 'return': <class 'bool'>}

This behavior ensures that :func:`get_annotations` returns annotations that
accurately reflect the signature of the partial or partialmethod object, as
determined by :func:`inspect.signature`.

If :func:`!get_annotations` cannot reliably determine which parameters are bound
(for example, if :func:`inspect.signature` raises an error), it will raise a
:exc:`TypeError` rather than returning incorrect annotations. This ensures that
you either get correct annotations or a clear error, never incorrect annotations:

.. doctest::

>>> from functools import partial
>>> import inspect
>>> def func(a: int, b: str) -> bool:
... return True
>>> partial_func = partial(func, 1)
>>> # Simulate a case where signature inspection fails
>>> original_sig = inspect.signature
>>> def broken_signature(obj):
... if isinstance(obj, partial):
... raise ValueError("Cannot inspect signature")
... return original_sig(obj)
>>> inspect.signature = broken_signature
>>> try:
... get_annotations(partial_func)
... except TypeError as e:
... print(f"Got expected error: {e}")
... finally:
... inspect.signature = original_sig
Got expected error: Cannot compute annotations for ...: unable to determine signature

This design prevents the common error of returning annotations that include
parameters which have already been bound by the partial application.


Recipes
-------
Expand Down
13 changes: 13 additions & 0 deletions Doc/library/functools.rst
Original file line number Diff line number Diff line change
Expand Up @@ -825,3 +825,16 @@ have three read-only attributes:
callable, weak referenceable, and can have attributes. There are some important
differences. For instance, the :attr:`~definition.__name__` and :attr:`~definition.__doc__` attributes
are not created automatically.

However, :class:`partial` objects do support the :attr:`~object.__annotate__` protocol for
annotation introspection. When accessed, :attr:`!__annotate__` returns only the annotations
for parameters that have not been bound by the partial application, along with the return
annotation. This behavior is consistent with :func:`inspect.signature` and allows tools like
:func:`annotationlib.get_annotations` to work correctly with partial objects. See the
:mod:`annotationlib` module documentation for more information on working with annotations
on partial objects.

:class:`partialmethod` objects similarly support :attr:`~object.__annotate__` for unbound methods.

.. versionadded:: next
Added :attr:`~object.__annotate__` support to :class:`partial` and :class:`partialmethod`.
147 changes: 147 additions & 0 deletions Lib/functools.py
Original file line number Diff line number Diff line change
Expand Up @@ -467,6 +467,8 @@ def _method(cls_or_self, /, *args, **keywords):
return self.func(cls_or_self, *pto_args, *args, **keywords)
_method.__isabstractmethod__ = self.__isabstractmethod__
_method.__partialmethod__ = self
# Set __annotate__ to delegate to the partialmethod's __annotate__
_method.__annotate__ = self.__annotate__
return _method

def __get__(self, obj, cls=None):
Expand All @@ -492,6 +494,10 @@ def __get__(self, obj, cls=None):
def __isabstractmethod__(self):
return getattr(self.func, "__isabstractmethod__", False)

def __annotate__(self, format):
"""Return annotations for the partial method."""
return _partialmethod_annotate(self, format)

__class_getitem__ = classmethod(GenericAlias)


Expand All @@ -513,6 +519,147 @@ def _unwrap_partialmethod(func):
func = _unwrap_partial(func)
return func

def _partial_annotate(partial_obj, format):
"""Helper function to compute annotations for a partial object.

This is called by the __annotate__ descriptor defined in C.
Returns annotations for the wrapped function, but only for parameters
that haven't been bound by the partial application.
"""
import inspect
from annotationlib import get_annotations

# Get the wrapped function
func = partial_obj.func

# Get annotations from the wrapped function
func_annotations = get_annotations(func, format=format)

if not func_annotations:
return {}

# Get the signature to determine which parameters are bound
try:
sig = inspect.signature(partial_obj, annotation_format=format)
except (ValueError, TypeError) as e:
# If we can't get signature, we can't reliably determine which
# parameters are bound. Raise an error rather than returning
# incorrect annotations.
raise TypeError(
f"Cannot compute annotations for {partial_obj!r}: "
f"unable to determine signature"
) from e

# Build new annotations dict with only unbound parameters
# (parameters first, then return)
new_annotations = {}

# Only include annotations for parameters that still exist in partial's signature
for param_name in sig.parameters:
if param_name in func_annotations:
new_annotations[param_name] = func_annotations[param_name]

# Add return annotation at the end
if 'return' in func_annotations:
new_annotations['return'] = func_annotations['return']

return new_annotations

def _partialmethod_annotate(partialmethod_obj, format):
"""Helper function to compute annotations for a partialmethod object.

This is called when accessing annotations on an unbound partialmethod
(via the __partialmethod__ attribute).
Returns annotations for the wrapped function, but only for parameters
that haven't been bound by the partial application. The first parameter
(usually 'self' or 'cls') is kept since partialmethod is unbound.
"""
import inspect
from annotationlib import get_annotations

# Get the wrapped function
func = partialmethod_obj.func

# Get annotations from the wrapped function
func_annotations = get_annotations(func, format=format)

if not func_annotations:
return {}

# For partialmethod, we need to simulate the signature calculation
# The first parameter (self/cls) should remain, but bound args should be removed
try:
# Get the function signature
func_sig = inspect.signature(func, annotation_format=format)
func_params = list(func_sig.parameters.keys())

if not func_params:
return func_annotations

# Calculate which parameters are bound by the partialmethod
partial_args = partialmethod_obj.args or ()
partial_keywords = partialmethod_obj.keywords or {}

# Build new annotations dict in proper order
# (parameters first, then return)
new_annotations = {}

# The first parameter (self/cls) is always kept for unbound partialmethod
first_param = func_params[0]
if first_param in func_annotations:
new_annotations[first_param] = func_annotations[first_param]

# For partialmethod, positional args bind to parameters AFTER the first one
# So if func is (self, a, b, c) and partialmethod.args=(1,)
# Then 'self' stays, 'a' is bound, 'b' and 'c' remain

# We need to account for Placeholders which create "holes"
# For example: partialmethod(func, 1, Placeholder, 3) binds 'a' and 'c' but not 'b'

remaining_params = func_params[1:]

# Track which positions are filled by Placeholder
placeholder_positions = set()
for i, arg in enumerate(partial_args):
if arg is Placeholder:
placeholder_positions.add(i)

# Number of non-Placeholder positional args
# This doesn't directly tell us which params are bound due to Placeholders

for i, param_name in enumerate(remaining_params):
# Check if this position has a Placeholder
if i in placeholder_positions:
# This parameter is deferred by Placeholder, keep it
if param_name in func_annotations:
new_annotations[param_name] = func_annotations[param_name]
continue

# Check if this position is beyond the partial_args
if i >= len(partial_args):
# This parameter is not bound at all, keep it
if param_name in func_annotations:
new_annotations[param_name] = func_annotations[param_name]
continue

# Otherwise, this position is bound (not a Placeholder and within bounds)
# Skip it

# Add return annotation at the end
if 'return' in func_annotations:
new_annotations['return'] = func_annotations['return']

return new_annotations

except (ValueError, TypeError) as e:
# If we can't process the signature, we can't reliably determine
# which parameters are bound. Raise an error rather than returning
# incorrect annotations (which would include bound parameters).
raise TypeError(
f"Cannot compute annotations for {partialmethod_obj!r}: "
f"unable to determine which parameters are bound"
) from e

################################################################################
### LRU Cache function decorator
################################################################################
Expand Down
Loading
Loading