PEP 749: Add a FAKE_GLOBALS_VALUE format (#3991)

Co-authored-by: Carl Meyer <[email protected]>

Author: JelleZijlstra

peps/pep-0749.rst

@@ -27,7 +27,9 @@ specification:
 * We specify the behavior of wrapper objects that provide annotations, such as :py:func:`classmethod`
   and code that uses :py:func:`functools.wraps`.
 * There will not be a code flag for marking ``__annotate__`` functions
-  that can be run in a "fake globals" environment.
+  that can be run in a "fake globals" environment. Instead, we add a fourth format,
+  ``VALUE_WITH_FAKE_GLOBALS``, to allow third-party implementors of annotate functions to
+  indicate what formats they support.
 * Setting the ``__annotations__`` attribute directly will not affect the ``__annotate__`` attribute.
 * We add functionality to allow evaluating type alias values and type parameter bounds and defaults
   (which were added by :pep:`695` and :pep:`696`) using PEP 649-like semantics.
@@ -74,6 +76,11 @@ We suggest the following deprecation plan:
 
 - In Python 3.14, ``from __future__ import annotations`` will continue to work as it
   did before, converting annotations into strings.
+
+  - If the future import is active, the ``__annotate__`` function of objects with
+    annotations will return the annotations as strings when called with the ``VALUE``
+    format, reflecting the behavior of ``__annotations__``.
+
 - Sometime after the last release that did not support :pep:`649` semantics (expected to be 3.13)
   reaches its end-of-life, ``from __future__ import annotations`` is deprecated. Compiling
   any code that uses the future import will emit a :py:exc:`DeprecationWarning`. This will
@@ -194,7 +201,8 @@ The module will contain the following functionality:
 * ``Format``: an enum that contains the possible formats of annotations. This will
   replace the ``VALUE``, ``FORWARDREF``, and ``SOURCE`` formats in :pep:`649`.
   PEP 649 proposed to make these values global members of the :py:mod:`inspect`
-  module; we prefer to place them within an enum.
+  module; we prefer to place them within an enum. We propose to add a fourth format,
+  ``VALUE_WITH_FAKE_GLOBALS`` (see below).
 * ``ForwardRef``: a class representing a forward reference; it may be returned by
   ``get_annotations()`` when the format is ``FORWARDREF``. The existing class
   :py:class:`typing.ForwardRef` will become an alias of this class. Its members include:
@@ -251,6 +259,9 @@ What should this module be called? Some ideas:
   and ``from __future__ import annotations`` in the same module. The use of a common word
   as the name will make the module harder to search for. There is a PyPI package :pypi:`annotations`,
   but it had only a single release in 2015 and looks abandoned.
+- ``annotation`` (in the singular): Similar, but does not cause confusion with the future
+  import. There is an abandoned PyPI package :pypi:`annotation`, but it apparently never
+  released any artifacts.
 - ``annotools``: Analogous to :py:mod:`itertools` and :py:mod:`functools`, but "anno" is a less
   obvious abbreviation than "iter" or "func". As of this writing, there
   is no PyPI package with this name.
@@ -561,8 +572,8 @@ This approach would also mean that accessing ``.__annotations__`` on an instance
 of an annotated class no longer works. While this behavior is not documented,
 it is a long-standing feature of Python and is relied upon by some users.
 
-Remove code flag for marking ``__annotate__`` functions
-=======================================================
+Adding the ``VALUE_WITH_FAKE_GLOBALS`` format
+=============================================
 
 :pep:`649` specifies:
 
@@ -577,33 +588,46 @@ Remove code flag for marking ``__annotate__`` functions
     it's expected that only ``__annotate__`` methods generated
     by the Python compiler will set it.
 
-We have not found a need for this mechanism during our work to
-add :pep:`649` support to the standard library. While it is true
-that custom ``__annotate__`` functions may not work well with the
-"fake globals" environment, this technique is used only when the
-``__annotate__`` function raises :py:exc:`NotImplementedError` to
-signal that it does not support the requested format. However,
-manually implemented ``__annotate__`` functions are likely to support
-all three annotation formats; often, they will consist of a call to
-``annotationlib.call_annotate_function`` plus some transformation of the
-result.
-
-In addition, the proposed mechanism couples the implementation with
+However, this mechanism couples the implementation with
 low-level details of the code object. The code object flags are
 CPython-specific and the documentation :py:ref:`explicitly warns <inspect-module-co-flags>`
 against relying on the values.
 
+Larry Hastings suggested an alternative approach that does not
+rely on code flags: a fourth format, ``VALUE_WITH_FAKE_GLOBALS``.
+Compiler-generated annotate functions would support only the
+``VALUE`` and ``VALUE_WITH_FAKE_GLOBALS`` formats, both of which are
+implemented identically. The standard library would use the
+``VALUE_WITH_FAKE_GLOBALS`` format when invoking an annotate function
+in one of the special "fake globals" environments.
+
+This approach is useful as a forward-compatible mechanism for
+adding new annotation formats in the future. Users who manually
+write annotate functions should raise ``NotImplementedError`` if
+the ``VALUE_WITH_FAKE_GLOBALS`` format is requested, so the standard
+library will not call the manually written annotate function with
+"fake globals", which could have unpredictable results.
+
 Specification
 -------------
 
-The standard library will use the "fake globals" technique on any
-``__annotate__`` function that raises :py:exc:`NotImplementedError`
-when the requested format is not supported.
+An additional format, ``FAKE_GLOBALS_VALUE``, is added to the ``Format`` enum in the
+``annotationlib`` module, with value equal to 2. (As a result, the values of the
+other formats will shift relative to PEP 649: ``FORWARDREF`` will be 3 and ``SOURCE``
+will be 4.)
+
+Compiler-generated
+annotate functions will support this format and return the same value as
+they would return for the ``VALUE`` format. The standard library will pass
+this format to the ``__annotate__`` function when it is called in a "fake globals"
+environment, as used to implement the ``FORWARDREF`` and ``SOURCE`` formats.
+All public functions in the ``annotationlib`` module that accept a format
+argument will raise :py:exc:`NotImplementedError` if the format is ``FAKE_GLOBALS_VALUE``.
 
-Third-party code that implements ``__annotate__`` functions should either
-support all three annotation formats, or be prepared to handle the
-"fake globals" environment. This should be mentioned in the data model
-documentation for ``__annotate__``.
+Third-party code that implements ``__annotate__`` functions should raise
+:py:exc:`NotImplementedError` if the ``FAKE_GLOBALS_VALUE`` format is passed
+and the function is not prepared to be run in a "fake globals" environment.
+This should be mentioned in the data model documentation for ``__annotate__``.
 
 Effect of setting ``__annotations__``
 =====================================
@@ -754,7 +778,7 @@ Signature of ``__annotate__`` functions
   ``__annotate__(format: int) -> dict``
 
 However, using ``format`` as a parameter name could lead to collisions
-if an annotation uses a class named ``format``. The parameter should be
+if an annotation uses a symbol named ``format``. The parameter should be
 positional-only and have a name that cannot be a legal identifier in order
 to avoid this problem.
 
@@ -846,7 +870,8 @@ initial decisions, but the overall design is still his.
 
 I thank Carl Meyer and Alex Waygood for feedback on early drafts of this PEP. Alex Waygood,
 Alyssa Coghlan, and David Ellis provided insightful feedback and suggestions on the
-interaction between metaclasses and ``__annotations__``.
+interaction between metaclasses and ``__annotations__``. Larry Hastings also provided useful
+feedback on this PEP.
 
 Appendix
 ========