PEP 770: Remove mechanism for statically defining SBOM files (#4321)

Author: sethmlarson

peps/pep-0770.rst

@@ -153,26 +153,19 @@ can projects begin using the new features. Even with this careful approach it's
 not guaranteed that tools won't break on new metadata versions and fields.
 
 To avoid this delay, simplify overall how to include SBOMs, and to give
-flexibility to build backends and tools, this PEP proposes a new top-level table
-in ``pyproject.toml``, ``[dist-info.files]``, to safely add data to a Python
-package through a registry of reserved names that
-avoids the need for new metadata fields and versions.  This mechanism allows
-build backends and tools to begin using the features described in this PEP
-immediately after acceptance without the head-of-line blocking on other
-projects adopting the PEP.
-
-A new top-level table was chosen over using the ``[project]`` table because,
-as described in :pep:`621`, the ``[project]`` table is used for storing core
-metadata and this mechanism doesn't use core metadata.
+flexibility to build backends and tools, this PEP proposes using a subdirectory
+under ``.dist-info`` to safely add data to a Python package while avoiding the
+need for new metadata fields and versions. This mechanism allows build backends
+and tools to begin using the feature described in this PEP immediately after
+acceptance without the head-of-line blocking on other projects adopting the PEP.
 
 Storing files in the ``.dist-info`` or ``.data`` directory
 ----------------------------------------------------------
 
 There are two top-level directories in binary distributions where files beyond
 the software itself can be stored: ``.dist-info`` and ``.data``.
 This specification chose to use the ``.dist-info`` directory for storing
-subdirectories and files from the new ``[dist-info.files]`` top-level table
-for two reasons:
+subdirectories and files.
 
 Firstly, the ``.data`` directory has no corresponding location in the installed
 package, compared to ``.dist-info`` which does preserve the link between the
@@ -231,37 +224,18 @@ Specification
 
 The changes necessary to implement this PEP include:
 
-* Explicitly reserving all subdirectory names in the ``.dist-info`` directory.
-* A new registry of reserved subdirectory names in the ``.dist-info`` directory.
-* An optional top-level table, ``[dist-info.files]``, added to
-  `project source metadata <770-spec-project-source-metadata_>`_,
-* An optional ``sboms`` key in the new ``[dist-info.files]`` table,
+* Explicitly reserving the subdirectory ``.dist-info/sboms``.
 * `Additions <770-spec-project-formats_>`_ to the built distribution (wheel),
   and installed project specifications
 
 In addition to the above, an informational PEP will be created for tools
 consuming included SBOM documents and other Python package metadata to
 generate complete SBOM documents for Python packages.
 
-.. _770-spec-dist-info-subdirs:
-
-Reserving all subdirectory names in ``.dist-info``
---------------------------------------------------
-
-This PEP explicitly reserves all subdirectory names in the ``.dist-info``
-directory for future usage.
-
-Build backends MUST NOT create subdirectories in the ``.dist-info`` directory
-beyond the names in the `registry <770-spec-dist-info-subdirs-registry>`_
-to avoid collisions with future reserved names.
-
-Build frontends and publishing tools MAY warn users if any ``.dist-info``
-subdirectories aren't in the registry.
-
 .. _770-spec-dist-info-subdirs-registry:
 
-Registry of reserved ``.dist-info`` subdirectory names
-------------------------------------------------------
+Reserving the ``.dist-info/sboms`` directory
+--------------------------------------------
 
 This PEP introduces a new registry of reserved subdirectory names allowed in
 the ``.dist-info`` directory for the :term:`distribution archive`
@@ -278,88 +252,7 @@ Subdirectory name PEP / Standard
 ================= ==============
 
 See :ref:`770-backwards-compat` for a complete methodology for
-creating this initial set of values to avoid backwards incompatibility issues.
-
-.. _770-spec-project-source-metadata:
-
-Project source metadata
------------------------
-
-This PEP specifies changes to the project's source metadata
-in the ``pyproject.toml`` file:
-
-Add new ``[dist-info.files]`` table
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-A new optional ``[dist-info.files]`` table is added for specifying paths
-in the project source tree relative to ``pyproject.toml`` to file(s) which
-should be included in the built project to a subdirectory of ``.dist-info``.
-
-This new table has only one defined optional key: ``sboms``. The value of the
-``sboms`` key MUST be an array of valid glob patterns, as specified below:
-
-* Alphanumeric characters, underscores (``_``), hyphens (``-``) and dots (``.``)
-  MUST be matched verbatim.
-* Special glob characters: ``*``, ``?``, ``**`` and character ranges: ``[]``
-  containing only the verbatim matched characters MUST be supported. Within
-  ``[...]``, the hyphen indicates a locale-agnostic range (e.g. a-z, order based
-  on Unicode code points). Hyphens at the start or end are matched literally.
-* Path delimiters MUST be the forward slash character (``/``). Patterns are
-  relative to the directory containing ``pyproject.toml``, therefore the leading
-  slash character MUST NOT be used.
-* Parent directory indicators (``..``) MUST NOT be used.
-
-Any characters or character sequences not covered by this specification are
-invalid. Projects MUST NOT use such values. Tools consuming this field SHOULD
-reject invalid values with an error.
-
-Literal paths (e.g. ``bom.cdx.json``) are treated as valid globs which means
-they can also be defined.
-
-Build tools:
-
-* MUST treat each value in the array as a glob pattern, and MUST raise an error
-  if the pattern contains invalid glob syntax.
-* MUST include all files matched by a listed pattern in all distribution
-  archives under the ``.dist-info/sboms`` directory.
-* MUST raise an error if any individual user-specified pattern does not match
-  at least one file.
-
-If the ``sboms`` key is present and is set to a value of an empty array,
-then tools MUST NOT include any SBOM files and MUST NOT raise an error.
-
-Examples of valid SBOM files declarations:
-
-.. code-block:: toml
-
-    [dist-info.files]
-    sboms = ["bom.json"]
-
-    [dist-info.files]
-    sboms = ["sboms/openssl.cdx.json", "sboms/openssl.spdx.json"]
-
-    [dist-info.files]
-    sboms = ["sboms/*"]
-
-    [dist-info.files]
-    sboms = []
-
-Examples of invalid SBOM files declarations:
-
-.. code-block:: toml
-
-    [dist-info.files]
-    sboms = ["..\bom.json"]
-
-Reason: ``..`` must not be used. ``\\`` is an invalid path delimiter, ``/``
-must be used.
-
-.. code-block:: toml
-
-    [dist-info.files]
-    sboms = ["bom{.json*"]
-
-Reason: ``bom{.json*`` is not a valid glob.
+avoiding backwards incompatibilities with selecting this directory name.
 
 .. _770-spec-project-formats:
 
@@ -368,12 +261,6 @@ SBOM files in project formats
 
 A few additions will be made to the existing specifications.
 
-:term:`Project source trees <Project source tree>`
-  Per :ref:`770-spec-project-source-metadata` section, the
-  `Declaring Project Metadata specification <pyprojecttoml_>`__
-  will be updated to add the ``[dist-info.files]`` table
-  and optional ``sboms`` key.
-
 :term:`Built distributions <Built distribution>` (:term:`wheels <wheel>`)
   The wheel specification will be updated to add the new registry of reserved
   directory names and to reflect that if the ``.dist-info/sboms`` subdirectory
@@ -420,10 +307,10 @@ SBOM standards, versions, or fields.
 Backwards Compatibility
 =======================
 
-Reserved ``.dist-info`` subdirectories registry
------------------------------------------------
+Reserved ``.dist-info/sboms`` subdirectory
+------------------------------------------
 
-The new registry of reserved ``.dist-info`` subdirectories represents
+The new reserved ``.dist-info/sboms`` subdirectory represents
 a new reservation that wasn't previously documented, thus has the potential to
 break assumptions being made by already existing tools.
 
@@ -521,23 +408,6 @@ Typically tools can automatically annotate bundled dependencies when those
 dependencies come from a "packaging ecosystem" (such as PyPI, Linux distros,
 Crates.io, NPM, etc).
 
-For packages which cannot be automatically annotated and if the package author
-wishes to provide an SBOM the approach will be to generate or author SBOM files
-and then include those files using ``pyproject.toml``:
-
-.. code-block:: toml
-
-    [dist-info.files]
-    sboms = [
-       "sboms/bom.cdx.json"
-    ]
-
-For projects manually specifying an SBOM document the challenge will be
-keeping the document up-to-date. The CPython project has some
-`customized tooling <https://github.com/python/cpython/blob/main/Tools/build/generate_sbom.py>`__
-for this task, but it can likely be generalized into a tool reusable by other
-projects.
-
 What do SBOM tool authors need to know?
 ---------------------------------------
 
@@ -644,11 +514,12 @@ user-involvement or knowledge. Compare this situation to license files which
 are almost always static.
 
 The 639-style approach was ultimately dropped in favor of defining SBOMs simply
-by their presence in the ``.dist-info/sboms`` directory and using a new table in
-``pyproject.toml`` called ``[dist-info.files]`` to define SBOMs in source
-distributions. This approach allows users to specify static SBOM files while
-still empowering build backends and tools to add their own SBOM data without the
-static/dynamic conflict.
+by their presence in the ``.dist-info/sboms`` directory. This approach allows
+build backends and tools to add their own SBOM data without the static/dynamic
+conflict.
+
+A future PEP will define the process for statically defining SBOM files to be
+added to the ``.dist-info/sboms`` directory.
 
 Open Issues
 ===========
@@ -686,9 +557,8 @@ Acknowledgements
 ================
 
 Thanks to Karolina Surma for authoring and leading :pep:`639` to acceptance.
-This PEP's initial design was heavily inspired by :pep:`639` and the new
-"dist-info.files" mechanism generalizes 639's approach of using a subdirectory
-under ``.dist-info``.
+This PEP's initial design was heavily inspired by :pep:`639` and adopts a
+similar approach of using a subdirectory under ``.dist-info`` to store files.
 
 Copyright
 =========