PEP 735 compliance

[DimitriPapadopoulos]

Patch Notes

This pull request migrates the project's dependency management to comply with PEP-735 by refactoring the pyproject.toml file. It replaces the deprecated [project.optional-dependencies] section with the new standardized [dependency-groups] format, enhancing clarity and organization for managing development dependencies.

Comprehensive Table of Package Managers Supporting PEP 735

Package Manager	PEP 735 Support	Notes	Documentation Link
Pip	Yes	Officially supports dependency groups since version 25.1. Allows installation using the --dependency-groups option.	Pip Documentation
Pip-tools	Planned	There are ongoing discussions and an open issue (#2062) for implementing PEP 735 support in pip-tools.	Pip-tools GitHub Issue
Poetry	Planned	Discussions for implementing support are ongoing, with expected future updates aligning with PEP 735 standards.	Poetry GitHub Issue
Tox	In Development	Current proposals suggest integration with dependency groups, but not yet implemented.	Tox GitHub Issue
ReadTheDocs	In Discussion	Proposals are underway to incorporate support for dependency groups into documentation builds.	ReadTheDocs Discussion
UV	Yes	Added support in version 0.4.27, allowing users to utilize dependency groups effectively.	UV Documentation
Easy_install	No	Considered deprecated; no current support for PEP 735 due to migration to more modern tools.	Setuptools Documentation
Homebrew	No	Primarily focuses on system-level package management; does not support Python's PEP 735 specifications.	Homebrew Documentation
apt-get	No	Designed for managing system packages for Linux, and does not include support for Python-specific PEP 735 group mechanisms.	Apt Documentation

Original Description

Here's an explanation borrowed from astral-sh/uv#8981 (comment):

The main difference from that perspective is that optional-dependencies (PEP 621) are part of the published metadata for your package, while dependency-groups (PEP 735) are only visible when working with your package locally.

[reactive-firewall]

👍 LGTM;

☑️ Specifically, the alignment with PEP-735 is approved once pip-tools supports the feature!

Additional Comments:

• It might be worth holding off on releasing this until tox and poetry are updated too. 🤷
• BLOCKED until jazzband/pip-tools#2175 or equivalent is implemented.

[DimitriPapadopoulos]

I am not sure why PEP 735 support by pip-tools or poetry would be required. How are they related?

[DimitriPapadopoulos]

Got it: pip-compile is used to pin dependencies, and pip-compile is part of pip-tools.

Unfortunately, pip-compile cannot handle the dependencies of virtualenv ≥ 20.36.0:

  "filelock>=3.16.1,<4; python_version<'3.10'",
  "filelock>=3.20.1,<4; python_version>='3.10'",

With Python 3.9, it attempts to install the latest version of virtualenv (20.36.1), and in turn to install filelock 3.20.1, which fails of course, because filelock ≥ 3.20.1 is not compatible with Python 3.9. That's one reason why I suggest moving to uv instead in #303. Another reason is that uv, like ruff, is blazingly fast.

[reactive-firewall]

Got it: pip-compile is used to pin dependencies, and pip-compile is part of pip-tools.

Unfortunately, pip-compile cannot handle the dependencies of virtualenv ≥ 20.36.0

Indeed, as you say. Also because you asked, and to clarify: poetry will not be a blocker for this.

With Python 3.9, it attempts to install the latest version of virtualenv (20.36.1), and in turn to install filelock 3.20.1, which fails of course, because filelock ≥ 3.20.1 is not compatible with Python 3.9.

And of course the bug is in virtualenv as you mentioned:

  "filelock>=3.16.1,<4; python_version<'3.10'",
  "filelock>=3.20.1,<4; python_version>='3.10'",

is obviously a mistake as it should be (see PEP-440)

 "filelock>=3.16.1,<3.20.1; python_version<'3.10'",
 "filelock>=3.20.1,<4; python_version>='3.10'",

🤦 I'm opening the bug report now.

[reactive-firewall]

Turns out the last version filelock to support python3.9 was 3.19.1 (e.g., not 3.20.0)

I've opened pypa/virtualenv#3037 and submitted a possible fix to the virtualenv side: pypa/virtualenv#3038

Hopefully that helps clear the way for this PR

[reactive-firewall]

💁 After talking with the people over on pip-tools they have re-affirmed they plan to release support for a --group option in an upcoming release.

---

I've become a pip-tools maintainer in the intervening year. My focus has been primarily on fixes and paying down tech debt -- new features like this have been more or less on hold.

I think it's likely to get included in one of the next few releases. We'll have to settle on exactly what we want in the annotation comments ("via"), but I think that's the only outstanding issue to address.

Originally posted by sirosen in #2062

---

From other discussions they seem a bit swamped over there (I've been shoed away for taking too much of their time)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.59%. Comparing base (50c3463) to head (1ba885b).
⚠️ Report is 19 commits behind head on dev-v5.6.

Additional details and impacted files

@@            Coverage Diff            @@
##           dev-v5.6     #299   +/-   ##
=========================================
  Coverage     99.59%   99.59%           
=========================================
  Files             1        1           
  Lines           491      491           
=========================================
  Hits            489      489           
  Misses            2        2           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.