PEP 753: Updates (#4010)

Signed-off-by: William Woodruff <[email protected]>
Co-authored-by: Paul Moore <[email protected]>

Author: woodruffw

peps/pep-0753.rst

@@ -15,13 +15,14 @@ Post-History: `26-Aug-2024 <https://discuss.python.org/t/core-metadata-should-ho
 Abstract
 ========
 
-This PEP recommends two discrete changes to handling of core metadata by
-indices, such as PyPI:
+This PEP recommends two discrete changes to the processing of core metadata by
+indices (such as PyPI) and other core metadata consumers:
 
 * Deprecation of the ``Home-page`` and ``Download-URL`` fields in favor of
   their ``Project-URL`` equivalents;
-* A set of conventions for normalizing and assigning semantics to
-  ``Project-URL`` labels.
+* A set of :ref:`conventions <conventions-for-labels>` for normalizing and
+  assigning semantics to ``Project-URL`` labels during consumer-side metadata
+  processing.
 
 Rationale and Motivation
 ========================
@@ -35,21 +36,21 @@ for expressing a package's relationship to external resources, via URLs:
 1. Metadata 1.0 introduced ``Home-page``, a single-use field containing
    a URL to the distribution's home page.
 
-   .. code-block::
+   .. code-block:: email
 
         Home-page: https://example.com/sampleproject
 
 2. Metadata 1.1 introduced ``Download-URL``, a complementary single-use field
    containing a URL suitable for downloading the current distribution.
 
-   .. code-block::
+   .. code-block:: email
 
         Download-URL: https://example.com/sampleproject/sampleproject-1.2.3.tar.gz
 
 3. Metadata 1.2 introduced ``Project-URL``, a **multiple-use** field containing
    a label-and-URL pair. Each label is free text conveying the URL's semantics.
 
-   .. code-block::
+   .. code-block:: email
 
         Project-URL: Homepage, https://example.com/sampleproject
         Project-URL: Download, https://example.com/sampleproject/sampleproject-1.2.3.tar.gz
@@ -93,9 +94,6 @@ This PEP stipulates the following for metadata producers:
 * When generating metadata 1.2 or later, producers **SHOULD** emit only
   ``Project-URL``, and **SHOULD NOT** emit ``Home-page`` or ``Download-URL``
   fields.
-* When generating ``Project-URL`` equivalents for ``Home-page`` and
-  ``Download-URL``, producers **SHOULD** use the
-  :ref:`label conventions <conventions-for-labels>` described below.
 
 These stipulations do not change the optionality of URL fields in core metadata.
 In other words, producers **MAY** choose to omit ``Project-URL`` entirely
@@ -106,6 +104,11 @@ or ``Download-URL``. However, see :ref:`future-considerations` for
 thoughts on how a new (as of yet unspecified) major core metadata version
 could complete the deprecation cycle via removal of these deprecated fields.
 
+Similarly, this PEP does **not** propose that metadata producers emit
+:ref:`normalized labels <label-normalization>`. Label normalization is performed
+solely on the processing and consumption side (i.e. within indices and other
+consumers of distribution metadata).
+
 .. _package-indices:
 
 Package indices
@@ -153,10 +156,13 @@ informal relationship between ``Home-page``, ``Download-URL``, and their
 
 This formalization has two parts:
 
-1. A set of rules for normalizing ``Project-URL`` labels;
+1. A set of rules for normalizing ``Project-URL`` labels during index-side
+   processing;
 2. A set of "well-known" normalized label values that indices may specialize
    URL presentation for.
 
+.. _label-normalization:
+
 Label normalization
 -------------------
 
@@ -190,14 +196,22 @@ normalization:
     "``Change_Log``", "``changelog``"
     "``What's New?``", "``whatsnew``"
 
-Metadata producers **SHOULD** emit the normalized form of a user
-specified label, but **MAY** choose to emit the un-normalized form so
-long as it adheres to the existing 32 character constraint.
+When processing distribution metadata, package indices **SHOULD** perform
+label normalization to determine if a given label is
+:ref:`well known <well-known>` for subsequent special processing.
+Labels that are not well-known **MUST** be processed in their un-normalized
+form.
+
+Normalization does **not** change pre-existing semantics around duplicated
+``Project-URL`` labels. In other words, normalization may result in duplicate
+labels in the project's metadata, but only in the same manner that was already permitted
+(since the :ref:`core metadata specification <packaging:core-metadata>` does
+not stipulate that labels are unique).
 
-Package indices **SHOULD NOT** use the normalized labels belonging to the set
-of well-known labels directly as UI elements (instead replacing them with
-appropriately capitalized text labels). Labels not belonging to the well-known
-set **MAY** be used directly as UI elements.
+Excerpted examples of normalized metadata fields are provided in
+:ref:`Appendix A <appendix-a>`.
+
+.. _well-known:
 
 Well-known labels
 -----------------
@@ -243,15 +257,9 @@ Indices **MAY** choose to use the human-readable equivalents suggested above
 in their UI elements, if appropriate. Alternatively, indices **MAY** choose
 their own appropriate human-readable equivalents for UI elements.
 
-Packagers and metadata producers **MAY** choose to use these well-known
-labels or their aliases to communicate specific URL intents to package indices
-and downstreams.
-
-Packagers and metadata producers **SHOULD** produce the normalized version
-of the well-known labels or their aliases in package metadata. Packaging tools
-**MUST NOT** transform between equivalent aliases, i.e.. **SHOULD**
-normalize ``GitHub`` to ``github`` but **MUST NOT** transform
-``github`` to ``source``.
+Packagers and metadata producers **MAY** choose to use any label that normalizes
+to these labels (or their aliases) to communicate specific URL intents to
+package indices and downstreams.
 
 Similarly, indices **MAY** choose to specialize their rendering or presentation
 of URLs with these labels, e.g. by presenting an appropriate icon or tooltip
@@ -264,12 +272,9 @@ for documentation hosting or issue tracking).
 
 This PEP recognizes that the list of well-known labels is unlikely to remain
 static, and that subsequent additions to it should not require the overhead
-associated with a formal PEP process or new metadata version. As the primary
-expected use case for this information is to control the way project URLs are
-displayed on the Python Package Index, this PEP proposes that the list above
-become a "living" list within PyPI's documentation (at time of writing, the
-documentation for influencing PyPI's URL display can be found
-`here <https://docs.pypi.org/project_metadata/#icons>`__).
+associated with a formal PEP process or new metadata version. As such,
+this PEP proposes that the list above become a "living" list within
+the :ref:`PyPA specifications <packaging:packaging-specifications>`.
 
 Backwards Compatibility
 =======================
@@ -349,6 +354,44 @@ already contains an informal description of PyPI's URL handling behavior.
 If this PEP is accepted, the authors of this PEP will coordinate to update
 and cross-link the resources mentioned above.
 
+.. _appendix-a:
+
+Appendix A: Label normalization examples
+========================================
+
+This appendix provides an illustrative excerpt of a distribution's
+metadata, before and after index-side processing:
+
+Before:
+
+.. code-block:: email
+
+    Project-URL: Home-page, https://example.com
+    Project-URL: Homepage, https://another.example.com
+    Project-URL: Source, https://github.com/example/example
+    Project-URL: GitHub, https://github.com/example/example
+    Project-URL: Another Service, https://custom.example.com
+
+
+After:
+
+.. code-block:: email
+
+  Project-URL: homepage, https://example.com
+  Project-URL: homepage, https://another.example.com
+  Project-URL: source, https://github.com/example/example
+  Project-URL: github, https://github.com/example/example
+  Project-URL: Another Service, https://custom.example.com
+
+In particular, observe:
+
+* Normalized duplicates are preserved (both ``Home-page`` and ``Homepage``
+  become ``homepage``);
+* ``Source`` and ``GitHub`` are both normalized into their respective forms,
+  but ``github`` is **not** transformed into ``source``.
+* ``Another Service`` is **not** normalized, since its normal form
+  (``anotherservice``) is not a :ref:`well-known label <well-known>`.
+
 Copyright
 =========