PEP 808: Partially Dynamic Metadata

.github/CODEOWNERS

@@ -680,6 +680,7 @@ peps/pep-0801.rst  @warsaw
 peps/pep-0802.rst  @AA-Turner
 peps/pep-0803.rst  @encukou
 peps/pep-0804.rst  @pradyunsg
+peps/pep-0808.rst  @FFY00
 # ...
 peps/pep-2026.rst  @hugovk
 # ...

peps/pep-0808.rst

@@ -0,0 +1,390 @@
+PEP: 808
+Title: Partially dynamic project metadata
+Author: Henry Schreiner <[email protected]>,
+        Cristian Le <[email protected]>
+Sponsor: Filipe Laíns <[email protected]>
+PEP-Delegate: Paul Moore <[email protected]>
+Status: Draft
+Type: Standards Track
+Topic: Packaging
+Created: 19-Sep-2025
+
+
+
+Abstract
+========
+
+This PEP relaxes the constraint on dynamic metadata listed in the ``[project]``
+section in ``pyproject.toml`` to allow the static portion of mixed metadata to
+be defined in the normal location if the field is a table or array by having
+the dynamic fields extend the static ones.
+
+This allows users to opt into allowing a backend to extend metadata while still
+keeping the static portions of the metadata defined in the standard location in
+``pyproject.toml``, and allows inspection tools to still be able to process the
+static portions of the metadata.
+
+
+Motivation
+==========
+
+In the core metadata specification originally set out in :pep:`621`, metadata
+can be specified in three ways. First, it can be listed in the ``[project]``
+table.  This makes it statically inferable, meaning any tool (not just the
+build backend) can reliably compute the value. Second, a field can be listed in
+the ``project.dynamic`` list, which allows the build backend to compute the
+value.  Finally, a value could be missing from both the ``project`` table and
+the ``project.dynamic`` list, in which case the matching metadata is guaranteed
+to be empty.
+
+This system provided two important benefits to Python packaging. A standard
+specification that all major backends have now adopted makes teaching much
+easier; a single tutorial is now sufficient to cover the metadata portion of
+configuring any backend. Users can now switch from a general purpose backend to
+a specialized backend without changing their static metadata. Tooling like
+schema validation tools can verify and catch configuration mistakes.
+
+The second benefit is improved support for static tools that read the source
+files looking for metadata. This is useful for dependency chain analysis, such
+as creating "used by" and "uses" graphs. It is used for code quality tooling to
+detect the minimum supported version of Python. It is used by cibuildwheel_ to
+automatically avoid building wheels that are not supported. It is not used,
+however, to avoid wheel builds when the SDist is available; that was addressed
+by METADATA 2.2, which a ``Dynamic`` field in the SDist metadata that lets a
+tool know if the metadata can change when making a wheel - this is an easy
+mistake to make due to the similarity of the names.
+
+Due to the rapidly increasing popularity of the project table, support from all
+major backends, and a rise of backends supporting complex compiled extensions,
+an issue with the restrictions applied in :pep:`621` is becoming more apparent.
+In PEP 621, the metadata choice is all-or-nothing; metadata must be completely
+static, or listed in the dynamic field and completely absent from the static
+definition. For the most common use cases, this is fine; there is little
+benefit to set the ``version`` statically if you are going to override it
+dynamically. If you are using a custom README processor to filter or modify the
+README for proper display, it's not a big deal to have to specify the
+configuration in a custom ``tool.*`` section. But there is a specific class of
+cases where the all-or-nothing approach is problematic: lists of items where
+the backend needs to add items are currently forced to be fully dynamically
+specified (that is, in a backend-specific configuration location). This causes
+both of the original benefits (standard location and static tooling support) to
+be lost.
+
+Rationale
+=========
+
+
+:pep:`621` includes the following statement:
+
+    In an earlier version of this PEP, tools were allowed to extend data for
+    fields. For instance, build back-ends could take the version number and add
+    a local version for when they built the wheel. Tools could also add more
+    trove classifiers for things like the license or supported Python versions.
+
+    In the end, though, it was thought better to start out stricter and
+    contemplate loosening how static the data could be considered based on
+    real-world usage.
+
+In this PEP, we are proposing a limited and explicit loosening of the
+restrictions on the ``[project]`` table and ``project.dynamic`` list.
+
+Every list and every table with arbitrary keys will now be allowed to be
+specified both statically, in the ``[project]`` table, and in the
+``project.dynamic`` list. If it is present in both places, the build backend
+can extend list items and add new keys, but not modify existing list items or
+strings.
+
+
+Use Cases
+---------
+
+There is an entire class of metadata fields where advanced use cases
+would really benefit from a relaxation of this rule. Here are some use
+cases that have come up:
+
+- Pinning dependency requirements when building the wheel. When building
+  PyTorch_ extensions, for example, the version you build with adds a constraint
+  to the wheel you create that is not present with the SDist.
+- Generating extra scripts from a build system (this is a currently proposed in
+  scikit-build-core_).
+- Adding entry points dynamically (validate-pyproject-schema-store_ could have
+  used this to generate an entry point for each schema present in the package.)
+- Adding dependencies or optional dependencies based on configuration (such as
+  making an all dependency, or reading dependencies from dependency-groups, for
+  example). Adding constraints can also be useful; pybind11_ uses adds a ``global``
+  extra that pins ``pybind11-global==<version>``, as both packages are in the
+  same repository and released in sync. toga_ is a collection of packages that
+  currently is unable to set any static dependencies due to the same sort of
+  pinning problem.
+- Adding classifiers; some backends can compute classifiers from other places
+  and inject them (Poetry_ being the best known example).
+- Adding license files to the wheel based on what libraries are linked in (this
+  is an active discussion in followup to :pep:`639`).
+- Adding SBom's when building - :pep:`770` had to remove the ``pyproject.toml``
+  field specifically because you _want_ the build tool to add these, so the
+  ``[project]`` table setting would be useless, you'd almost never be able to
+  use it.
+
+All of these use cases have a similar feature: they are adding something
+dynamically to a fixed list (possibly a narrower pin for the dependency case).
+
+You can implement these today, but it requires providing a completely separate
+configuration location for the non-extended portion, and static analysis tools
+lose the ability to detect anything. Since the current solution is to move all
+the metadata out of the standard field, this proposal will increase the
+availability of metadata for static tooling.
+
+
+Example: pinning
+----------------
+
+For example, let's say you want to allow an imaginary build backend
+(``my-build-backend``) to pin to the supported build of PyTorch_.  Before this
+PEP, you could do this:
+
+.. code-block:: toml
+
+    [project]
+    dynamic = ["dependencies"]
+
+    [tool.my-build-backend]
+    original-dependencies = ["torch", "packaging"]
+    pin-to-build-versions = ["torch=={exact}"]
+
+Which would effectively expand to the following SDist metadata:
+
+.. code-block:: text
+
+   Dynamic: Requires-Dist
+   Requires-Dist: packaging
+   Requires-Dist: torch
+
+Which would then could make a wheel with this:
+
+.. code-block:: text
+
+   Requires-Dist: packaging
+   Requires-Dist: torch
+   Requires-Dist: torch==2.8.0
+
+Static tooling no longer can tell that ``torch`` and ``packaging`` are runtime
+dependencies, and the build backend had to duplicate the dependency table,
+making it harder for users to learn and read; the standardized place proposed
+by :pep:`621` and adopted by all major build backends is lost.
+
+With this PEP, this could now be specified like this:
+
+.. code-block:: toml
+
+    [project]
+    dependencies = ["torch", "packaging"]
+    dynamic = ["dependencies"]
+
+    [tool.my-build-backend]
+    pin-to-build-versions = ["torch=={exact}"]
+
+Static tooling can now detect the static dependencies, and the build backend no
+longer needs to create and document a new location for the standard
+``project.dependencies`` field (the ``original-dependencies`` field above, for
+example).
+
+
+
+Future Updates
+--------------
+
+Loosening this rule to allow purely additive metadata should address many of
+the use cases that have been seen in practice. If further changes are needed,
+this can be revisited in a future PEP; this PEP neither recommends or precludes
+future updates like this.
+
+Terminology
+===========
+
+The keywords "MUST", "MUST NOT", "REQUIRED",
+"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL"
+in this document are to be interpreted as described in :rfc:`2119`.
+
+Specification
+=============
+
+Any field that is comprised of a list or a table with arbitrary entries will
+now be allowed to be present in both the ``[project]`` table and the
+``project.dynamic`` list. If a field is present in both places, then the build
+backend is allowed to insert entries into the list or table, but not remove
+entries, reorder entries, or modify the entries. Tables of arrays allow adding
+a new table entry or extending an existing array according to the rules above.
+
+The fields that are arrays or tables with arbitrary entries are:
+
+* ``authors``, ``maintainers``: New author tables can be added to the list.
+  Existing authors cannot be modified (list of tables with pre-defined keys).
+* ``classifiers``: Classifiers can be added to the list.
+* ``dependencies``: New dependencies can be added, including more tightly
+  constrained existing dependencies.
+* ``entry-points``: Entry points can be added, to either new or existing
+  groups. Existing entry points cannot be changed or removed.
+* ``scripts``, ``gui-scripts``: New scripts can be added. Existing ones cannot
+  be changed or removed.
+* ``keywords``: Keywords can be added to the list.
+* ``license-files``: Files can be added to the list.
+* ``optional-dependencies``: A new extra or new items can be added to a
+  existing extra.
+* ``urls``: New urls can be added. Existing ones cannot be changed or removed.
+
+To add items, users must opt-in by listing the field in ``dynamic``; without
+that, the metadata continues to be entirely static.
+
+A backend SHOULD error if a field is specified and it does not support
+extending that field, to protect against possible user error. We recommend
+being as strict as possible to avoid unnecessary entries in the ``dynamic``
+list.
+
+Static analysis tools, when detecting a field is both specified and in the
+``project.dynamic`` array, SHOULD assume the field is incomplete, allowing for
+new entries to be present when the package is built.
+
+The ``Dynamic`` field, as specified in :pep:`643`, is unaffected by this PEP,
+and backends can continue to fill it as they chose. However, a backend MUST
+ensure that both the SDist and the wheel metadata include the static metadata
+portion of the project table.
+
+Reference Implementation
+========================
+
+The choice to support dynamic metadata for each field is already left up to
+backends, and this PEP simply relaxes restrictions on what a backend is allowed
+to do with dynamic metadata.
+
+The pyproject-metadata_ project, which is used by
+several build backends, will need to modify the correctness check to account
+for the possible extensions; this is in `a draft PR <pyprojectmetdatapr>`__.
+
+The dynamic-metadata_ project, which provides a plugin
+system that backends can use to share dynamic metadata plugins, was designed to
+allow this possibility, and a similar PR to the one above will allow additive
+metadata.
+
+Backwards Compatibility
+=======================
+
+This does not affect any existing ``pyproject.toml``'s, since this was strictly
+not allowed before this PEP.
+
+When users adopt this in a ``pyproject.toml``, the backend must support it; an
+error will be correctly generated if it doesn't following the previous
+standard.  Frontends were never required to throw an error, though some
+frontends may need to be updated to benefit from the partially static metadata.
+Some frontends and other tooling may need updating, such as schema
+validation, just like other ``pyproject.toml`` PEPs.
+
+Using metadata from SDists or wheels is unaffected. The METADATA version does
+not need to be incremented.
+
+Security Implications
+=====================
+
+There are no security concerns that are not already present, as this just adds
+a static component to existing dynamic metadata support.
+
+How to Teach This
+=================
+
+The current guides that state metadata must not be listed in both ``[project]``
+and ``project.dynamic`` can be updated to say that some fields can be extended
+by ``project.dynamic``. Since dynamic metadata is already an advanced concept,
+this will likely not affect most existing tutorial material aimed at
+introductory packaging.
+
+The ``pyproject.toml`` `specification <pyprojectspec>`__ will be updated to
+include the behavior of fields when specified and also listed in the dynamic
+field.
+
+It should also be noted that specifying something in dynamic will require any
+tool that requires the full metadata to invoke the backend even if it is
+partially statically specified, so it should not be used unless necessary.
+
+
+Rejected Ideas
+==============
+
+Special case some fields without adding dynamic
+-----------------------------------------------
+
+This has come up specifically for the pinning build dependency use case, but
+could also be applied to more of the use cases listed. This would not cover all
+the use cases seen, though, and an explicit, opt-in approach is better for
+static tooling.
+
+
+Include string fields
+---------------------
+
+Some string fields could also be extended. Most notably, the ``license`` field
+would benefit from being extendable, and due to the semantics of SPDX
+expressions, extension could be defined through ``AND``. This was not added to
+this PEP because that would require individual fields to have custom semantics.
+
+The other string fields, namely ``version`` and ``requires-python`` (``name``
+is not allowed to be specified dynamically), have less reason to be extended.
+Fixed key tables, like the deprecated ``license.text``/``license.file`` or
+``readme.text``/``readme.file`` also have no clear benefit being partially
+dynamic.
+
+
+Fully remove restrictions on backends
+-------------------------------------
+
+Another option would be to simply allow backends to do whatever they wanted if
+a field is statically defined and in the dynamic array. This would sacrifice
+the ability for static tooling to infer anything about the field, and could
+potentially confuse users by allowing the backend to ignore or change what they
+entered. This is not worse than the status quo for static tooling and dynamic
+metadata, but the current proposal improves the ability of static tooling to
+infer some things about dynamic fields. Knowing some of the dependencies is
+better for most applications than not knowing anything about the dependencies,
+for example.
+
+Allow simplifications
+---------------------
+
+An earlier draft of this PEP had a clause allowing backends to simplify some
+types of fields; most notably dependency specifiers would have allowed
+"tightening", such as ``torch`` being replaced by ``torch>=1.2``, for example.
+. This was removed due to it being impossible to ensure a variation will
+resolve identically on all resolvers within the current specification, and to
+simplify the contract with backends. Any other simplifications would be purely
+cosmetic, and so were left out. The order in the current PEP is now required to
+match the original static metadata, with the dynamic portion only allowing
+insertions.
+
+
+Add a general mechanism to specify dynamic-metadata
+---------------------------------------------------
+
+This PEP does not cover methods to specify dynamic metadata; that continues to
+be entirely up to the backend. An earlier draft proposal did this, but it was
+deemed better to develop that as a library (dynamic-metadata_, for the curious)
+instead. This may be revisited in the future.
+
+References
+==========
+
+.. _cibuildwheel: https://cibuildwheel.pypa.io
+.. _pyprojectspec: https://packaging.python.org/en/latest/specifications/pyproject-toml
+.. _pyproject-metadata: https://github.com/pypa/pyproject-metadata
+.. _pyprojectmetadatapr: https://github.com/pypa/pyproject-metadata/pull/241
+.. _dynamic-metadata: https://github.com/scikit-build/dynamic-metadata
+.. _PyTorch: https://pytorch.org/
+.. _scikit-build-core: https://github.com/scikit-build/scikit-build-core
+.. _validate-pyproject-schema-store: https://pypi.org/project/validate-pyproject-schema-store/
+.. _pybind11: https://github.com/pybind/pybind11
+.. _Poetry: https://python-poetry.org/
+.. _setuptools: https://github.com/pypa/setuptools
+.. _toga: https://github.com/beeware/toga
+
+Copyright
+=========
+
+This document is placed in the public domain or under the
+CC0-1.0-Universal license, whichever is more permissive.