PEP 808: Partially Dynamic Metadata

[henryiii]

This PEP proposes an update to the pyproject.toml dynamic metadata specification to enable partially dynamic metadata, where some fields can be partially specified and also dynamically extended.

Pre-PEP thread: https://discuss.python.org/t/partially-dynamic-project-metadata-proposal-pre-pep/88608

Filipe (@FFY00) has graciously agreed to be my sponsor for this PEP.

Basic requirements (all PEP Types)

• Read and followed PEP 1 & PEP 12
• File created from the latest PEP template
• PEP has next available number, & set in filename (pep-NNNN.rst), PR title (PEP 123: <Title of PEP>) and PEP header
• Title clearly, accurately and concisely describes the content in 79 characters or less
• Core dev/PEP editor listed as Author or Sponsor, and formally confirmed their approval
• Author, Status (Draft), Type and Created headers filled out correctly
• PEP-Delegate, Topic, Requires and Replaces headers completed if appropriate
• Required sections included
  • Abstract (first section)
  • Copyright (last section; exact wording from template required)
• Code is well-formatted (PEP 7/PEP 8) and is in code blocks, with the right lexer names if non-Python
• PEP builds with no warnings, pre-commit checks pass and content displays as intended in the rendered HTML
• Authors/sponsor added to .github/CODEOWNERS for the PEP

Standards Track requirements

• PEP topic discussed in a suitable venue with general agreement that a PEP is appropriate
• Suggested sections included (unless not applicable)
  • Motivation
  • Rationale
  • Specification
  • Backwards Compatibility
  • Security Implications
  • How to Teach This
  • Reference Implementation
  • Rejected Ideas
  • Open Issues
• Python-Version set to valid (pre-beta) future Python version, if relevant
• Any project stated in the PEP as supporting/endorsing/benefiting from the PEP formally confirmed such
• Right before or after initial merging, PEP discussion thread created and linked to in Discussions-To and Post-History

---

📚 Documentation preview 📚: https://pep-previews--4598.org.readthedocs.build/pep-0808/

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[henryiii]

@LecrisUT, you need to sign the CLA, see above.

[pfmoore]

As I've mentioned before, I strongly disagree with special-casing particular fields. The PEP as a whole should be written in terms of combining basic data structures - i.e., tables and arrays.

If license needs extending, then a separate PEP allowing license to be an array, and then the semantics of extending follow from the basic rules, plus the rule on how a license expressed as an array is interpreted.

[pfmoore]

As noted above, this should be handled in a separate PEP, extending license to allow an array (semantics defined as combining the array elements with AND).

[eli-schwartz]

Permitting array semantics for license is "technically well defined" in this example since you define it as meaning AND. But unfortunately one of the major reasons to define SPDX licenses as a string is to reduce confusion when people are (for various reasons) not clear on how joining multiple licenses should work, and end up inputting incorrect information.

Whether it makes sense that it confuses people or not, is beside the point IMO. I think that this is a potential risk of handling this as a generic extension.

It's much less risky to have this be part of build backends, with a scope hopefully limited to "the build backend dynamically adds vendored/statically-linked dependencies". Backends can be designed to reliably compute this.

It's not a major criticism I guess. But it does make me a bit nervous. I don't really see why special cases should be a problem.

[pfmoore]

The point is that the semantics of adding new entries to an array is unambiguous and obvious. In contrast, the semantics of "adding a new license" need to be specified, and getting into that sort of complex debate is not in scope for this PEP.

If "[A, B] means (A) AND (B)" isn't the right semantics for an array of license expressions, then that's something for a PEP focused on license expression semantics and the meaning of the License metadata item and its associated license key in pyproject.toml to sort out. Not something we need to debate here.

Basically, I don't want this PEP to be blocked on a debate about license semantics, and I don't want to set the (bad) precedent that future PEPs adding new metadata (or updating existing metadata like dependencies) can add another bunch of custom semantics to the mechanism defined in this PEP.

[henryiii]

Supporting licenses is one of the main reasons for this PEP, and is intended to support a new licensing PEP that will expand the license field into multiple fields, by the authors of PEP 639 like @CAM-Gerlach. This would allow the number of new fields in that new PEP to be reduced, since some of them amount to "add licenses of things you include" that really would better be covered by this general mechanism.

I really don't like the idea of duplicating the number of ways to do exactly the same thing for the users (100K+ pyproject.tomls) just to give the backends (10 or so) a slightly easier time. Your proposal means there now are three ways to write this:

license = "MIT AND BSD"
license = ["MIT AND BSD"]
license = ["MIT", "BSD"]

These are all equivalent, except the first one won't allow you to add it to the dynamic list, while the others do. IMO, this is terrible UX and much harder to teach.

I think it's better to simply view "license" as a custom language (it's well defined), and that custom language supports addition via AND, so it should be included, just like lists and tables. We can reduce the "special case" language.

If someone adds a new custom field that encapsulates a mini language, then they should also specify if this supports being in dynamic; just like today you need to specify if a new field supports being in dynamic at all (name does not, for example). While you don't need to specify anything if it's just an arbitrary string.

I'm fine to either:

a) rework this to avoid "special case", and keep it more focused on the fact license is not an arbitrary string.
b) remove it for now, since a license PEP is coming. But I'd recommend that PEP extend strings as is described here, not allow multiple ways to write licenses.

[pfmoore]

I'm sorry, but this is a dealbreaker for me. I really don't want a PEP that specifies a general approach to extending metadata to have special cases for particular fields, with their own semantics. I don't think it's good for user understanding, and I think it will be a source of hard to understand bugs. IMO, if a user specifies static metadata and dynamic in pyproject.toml, then what that means should be understandable based solely on the data types in the TOML language. It should not require any field-specific understanding from the user or the build backend.

I think you're exaggerating the issue with having ["MIT", "BSD"] mean the same as "MIT AND BSD". It's just two ways of saying the same thing. You could also write "BSD AND MIT", or "(BSD) AND (MIT)", or any of a number of other forms - and more with complex combinations, the possibilities explode. That's just a consequence of having a complex expression language for a field.

Your option (b) is basically what I'm proposing, but I'll fight you just as hard on that license PEP. So you're not gaining anything beyond a delay in having the fight 🙂

This is a sufficiently important discussion that I don't think it's appropriate to continue it here. Can wse please move it to Discourse, where the rest of the community can more easily participate?

[hugovk]

• Core dev/PEP editor listed as Author or Sponsor, and formally confirmed their approval

@FFY00 Please could you confirm your sponsorship?

[FFY00]

@hugovk, I confirm 😊