Implement PEP 639

[abravalheri]

Summary of changes

This PR is a "mega PR" that simply make sure all the steps to implement PEP 639 are added at the same time to the main branch.
This is done to avoid problems with incomplete changes:

• We know that tools are being designed and implement PEP 639 compatibility
• We know existing tools know how to understand the current implementation in setuptools
• We don't know how tools would react to something that is not quite a PEP 639 implementation or the exact existing implementation in setuptools.

All the PR are previous reviewed individually.

• Update validate-pyproject to 0.23.0 #4734
• Store license-files in licenses subfolder #4728
• Add initial support for license expression (PEP 639) #4706
• Bump core metadata version to 2.4 #4830
• Prevent deprecated license classifiers from being written to core metadata #4833
• Ensure _apply_pyproject sets field on dist.metadata object not on dist #4834

Good to have according to #4829 (comment):

• The PEP specifies that tools should reject any patterns which contain not-approved chars.

• Partially: Add initial warnings regarding license-files glob patterns. #4838

• Furthermore, it should be an error if a pattern doesn't match any files.

• Partially: Add initial warnings regarding license-files glob patterns. #4838

• Tools should also validate that the license files themselves are utf-8 encoded. Although I don't think that's a priority tbh.

• Currently missing but not blocking

• Lastly, we should probably provide some kind of transition path from tool.setuptools.license-files to project.license-files. Be it as simple as rejecting both together (in which case tool.setuptools.license-files is used) or emitting a warning that tool.setuptools.license-files is deprecated in favor of project.license-files.

• Deprecate tools.setuptools.license-files #4837

Compatibility with other peps:

• Improve interop between Dynamic (METADATA) and dynamic (pyproject.toml) with license/license_expression/license_files #4842

NOTE: I believe that we can simply fast forward the Core Metadata to 2.4 considering that:

• Setuptools emits normalised extra names for Core Metadata:
  https://github.com/pypa/setuptools/blob/v75.8.0/setuptools/_core_metadata.py#L231-L237
  https://github.com/pypa/setuptools/blob/v75.8.0/setuptools/_normalization.py#L100-L107
• Setuptools defers to packaging.requirements.Requirement.__str__ the requirement normalisation, which should also normalise the extra marker¹

https://github.com/pypa/setuptools/blob/v75.8.0/setuptools/_core_metadata.py#L236-L250

Closes

Pull Request Checklist

• Changes have tests
• News fragment added in newsfragments/.
  (See documentation for details)

Footnotes

1. Currently there is a problem with packaging implementation https://github.com/pypa/packaging/issues/845, but for the sake of advancing the setuptools implementation to catch up with the standards we can consider this a "well known bug" that would be solved in the context of pypa/packaging. ↩

[cdce8p]

Opened #4830 with the metadata version bump. I can rebase it after #4706 is merged into the feature branch.

[abravalheri]

Thank you very much @cdce8p!

It seems that a new error with installing tox on cygwin appeared on the CI. I will try to check if it exists in the main branch as it is probably unrelated.

[cdce8p]

One area in which the implementation could still be improved is probably the license-files handling. https://peps.python.org/pep-0639/#add-license-files-key

• The PEP specifies that tools should reject any patterns which contain not-approved chars.
• Furthermore, it should be an error if a pattern doesn't match any files.
• Tools should also validate that the license files themselves are utf-8 encoded. Although I don't think that's a priority tbh.
• Lastly, we should probably provide some kind of transition path from tool.setuptools.license-files to project.license-files. Be it as simple as rejecting both together (in which case tool.setuptools.license-files is used) or emitting a warning that tool.setuptools.license-files is deprecated in favor of project.license-files.

There is also the matter of setup.py and setup.cfg files. My initial intent was to require pyproject.toml for PEP 639. However I haven't tested what happens if one of the setup files overwrites the project metadata.

Side note: Adding support for license expressions to setup.py or setup.cfg is probably a lost cause in itself, as these use a license key already.

--
This looks like a larger list than it is. In particular, they are mostly quality-of-life improvements. So I wouldn't consider them a necessity for the initial implementation.

[abravalheri]

Thank you for the review @cdce8p.

Let's make a checkbox:

• The PEP specifies that tools should reject any patterns which contain not-approved chars.
• Furthermore, it should be an error if a pattern doesn't match any files.
• Tools should also validate that the license files themselves are utf-8 encoded. Although I don't think that's a priority tbh.
• Lastly, we should probably provide some kind of transition path from tool.setuptools.license-files to project.license-files. Be it as simple as rejecting both together (in which case tool.setuptools.license-files is used) or emitting a warning that tool.setuptools.license-files is deprecated in favor of project.license-files.

I would like to tackle at least the last of this points before releasing because we can introduce all the related deprecation warnings/breaking errors at once.

Side note: Adding support for license expressions to setup.py or setup.cfg is probably a lost cause in itself, as these use a license key already.

I agree that there is not much point on updating setup.cfg or setup.py support... although it might be the case if someone sets license_expression = ... there it will just work (remains untested).

[abravalheri]

This week we had some friction with a different feature that was "queued for release", and next week I will not be around to help with the release/fixing reports from users. So the earliest we could do is in the week after the next one.

[cdce8p]

next week I will not be around to help with the release/fixing reports from users. So the earliest we could do is in the week after the next one.

I could help out if I see something. Tbh I think the risk here is fairly well contained. As long as we don't break existing configs which this branch shouldn't do, we're likely safe. Anything else which doesn't work 100% could be fixed afterwards as well, once there is more time.

[abravalheri]

@jaraco, would be OK if I create a release merging this implementation some time tomorrow? Or should I wait time more until we stabilise the change in distutils.compilers?

I would like to cut a release including this PR (#4829) + #4870 + #4869

(The cygwin error in the CI is already fixed on main)

[abravalheri]

OK, there has been a couple of other releases this week.
So it might be prudent to wait another one.

[jaraco]

No objections to any changes On Wed, Mar 12, 2025 at 06:10, Anderson Bravalheri ***@***.******@***.***>> wrote: OK, there has been a couple of other releases this week. So it might be prudent to wait another one. — Reply to this email directly, view it on GitHub<#4829 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AACLLAWZTVSTVCUDHHGAUJT2UABXTAVCNFSM6AAAAABXJJLRAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMJXGM2DEMJQHE>. You are receiving this because you were mentioned.Message ID: ***@***.***> [abravalheri]abravalheri left a comment (pypa/setuptools#4829)<#4829 (comment)> OK, there has been a couple of other releases this week. So it might be prudent to wait another one. — Reply to this email directly, view it on GitHub<#4829 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AACLLAWZTVSTVCUDHHGAUJT2UABXTAVCNFSM6AAAAABXJJLRAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMJXGM2DEMJQHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[abravalheri]

I plan to work on this release this week (hopefully tomorrow).

I understand that there is still some sort of instability due to recent distutils changes (e.g. #4885), but hopefully they affect a small portion of the userbase (¿?) ... and we have been holding back for a while.