feat: Support PEP 735 dependency groups

[finswimmer]

Required-by: python-poetry/poetry#10130
Relates-to: python-poetry/poetry#9751

• Added tests for changed code.
• Updated documentation for changed code.

Summary by Sourcery

New Features:

• Add support for PEP 735 dependency groups.

Summary by Sourcery

Implement support for PEP 735 dependency groups in Poetry core, merging the new [dependency-groups] table with existing tool.poetry.group settings, propagating group metadata to dependencies, and adding comprehensive validation.

New Features:

• Add parsing and handling of the [dependency-groups] section in pyproject.toml to define named dependency groups.
• Support include-group directives to compose groups and merge them with tool.poetry.group configurations.

Enhancements:

• Extend create_from_pep_508 to accept and propagate dependency group metadata to all dependency types.
• Refactor Factory._configure_package_dependencies to unify PEP 735 and legacy group definitions and normalize group names.

Tests:

• Add integration tests for group creation, group include behavior, error cases (invalid, duplicated, missing include), and cycle detection.
• Include fixture projects showcasing both traditional and PEP 735 group definitions.

[sourcery-ai]

Reviewer's Guide

This PR implements full PEP 735 support by extending the Factory to parse and normalize a new [dependency-groups] section alongside legacy tool.poetry.group tables, propagating group metadata through PEP 508 and file/dir dependency creation, enforcing schema validation with duplicate and cycle detection, and adding comprehensive tests to cover both valid and error scenarios.

ER diagram for dependency-groups and group includes

erDiagram
    DEPENDENCY_GROUPS {
        string name PK
        bool optional
    }
    DEPENDENCY {
        string name PK
        string version
        string[] groups
    }
    DEPENDENCY_GROUPS ||--o{ DEPENDENCY : contains
    DEPENDENCY_GROUPS ||--o{ DEPENDENCY_GROUPS : includes
    DEPENDENCY }o--|| DEPENDENCY_GROUPS : belongs_to

Loading

Class diagram for updated Dependency creation and group propagation

classDiagram
    class Dependency {
        +with_groups(groups: Iterable[str]) Dependency
        +create_from_pep_508(name: str, relative_to: Path|None = None, groups: Iterable[str]|None = None) Dependency
    }
    class FileDependency {
        <<Dependency>>
        +groups: Iterable[str]|None
    }
    class DirectoryDependency {
        <<Dependency>>
        +groups: Iterable[str]|None
    }
    class VCSDependency {
        <<Dependency>>
        +groups: Iterable[str]|None
    }
    class URLDependency {
        <<Dependency>>
        +groups: Iterable[str]|None
    }
    Dependency <|-- FileDependency
    Dependency <|-- DirectoryDependency
    Dependency <|-- VCSDependency
    Dependency <|-- URLDependency

    Dependency o-- "*" groups: Iterable[str]
    FileDependency o-- "*" groups: Iterable[str]
    DirectoryDependency o-- "*" groups: Iterable[str]
    VCSDependency o-- "*" groups: Iterable[str]
    URLDependency o-- "*" groups: Iterable[str]

Loading

Class diagram for Factory dependency group handling

classDiagram
    class Factory {
        +_add_package_pep735_group_dependencies(package, group, dependencies) list[str]
        +_add_package_poetry_group_dependencies(package, group, dependencies)
        +_configure_package_dependencies(package, project, tool_poetry, dependency_groups, with_groups)
        +_validate_dependency_groups(toml_data, result)
    }
    class ProjectPackage {
        +add_dependency_group(group: DependencyGroup)
        +has_dependency_group(name: str) bool
        +dependency_group(name: str) DependencyGroup
    }
    class DependencyGroup {
        +add_dependency(dep: Dependency)
        +include_dependency_group(group: DependencyGroup)
        +optional: bool
        +name: str
    }
    Factory o-- ProjectPackage
    ProjectPackage o-- "*" DependencyGroup
    DependencyGroup o-- "*" Dependency
    DependencyGroup o-- "*" DependencyGroup : includes

Loading

Flow diagram for dependency group normalization and cycle detection

flowchart TD
    A[Read dependency-groups and tool.poetry.group sections] --> B[Normalize group names]
    B --> C[Check for duplicate group names]
    C --> D[Build group include graph]
    D --> E[Detect cycles in group includes]
    E --> F[Report errors if duplicates or cycles found]
    F --> G[Proceed with group creation if valid]

Loading

File-Level Changes

Change	Details	Files
Extend Factory to parse and integrate PEP 735 dependency-groups	Introduce _add_package_pep735_group_dependencies to consume group-specific dependencies and collect include directives Modify configure_package and _configure_package_dependencies to accept a dependency_groups argument Normalize tool.poetry.group names, create DependencyGroup instances for both PEP 735 and legacy groups, and merge include-groups relationships	src/poetry/core/factory.py
Propagate group metadata in dependency creation	Add optional groups parameter to Dependency.create_from_pep_508 Pass groups through VCSDependency, URLDependency, FileDependency, DirectoryDependency constructors Include group tags in the default Dependency instantiation	src/poetry/core/packages/dependency.py
Validate dependency-groups schema and detect duplicates/cycles	Invoke JSON schema validation on [dependency-groups] and merge errors into the validation result Implement _validate_dependency_groups to identify normalized-name collisions and self- or cross-cycle includes Update error reporting to reference the dependency-groups path correctly	src/poetry/core/factory.py
Add tests and fixtures for PEP 735 dependency-groups	Add parametrized tests in test_factory.py for valid group parsing and metadata propagation Add failure tests covering invalid definitions: missing includes, duplicate names, and cyclic includes Provide sample_project_with_groups and fixture pyproject.toml files for new and legacy formats	tests/test_factory.py tests/fixtures/sample_project_with_groups/pyproject.toml tests/fixtures/sample_project_with_groups_new/pyproject.toml tests/fixtures/project_with_invalid_dependency_groups/pyproject.toml tests/fixtures/project_with_dependency_groups_simple_cycle/pyproject.toml tests/fixtures/project_with_dependency_groups_complex_cycle/pyproject.toml tests/fixtures/project_with_duplicated_dependency_groups/pyproject.toml tests/fixtures/project_with_dependency_groups_missing_include/pyproject.toml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[finswimmer]

@sourcery-ai review

[sourcery-ai]

Hey @finswimmer - I've reviewed your changes - here's some feedback:

Overall Comments:

• Please add documentation for the new dependency groups feature - this is a significant change that needs to be well documented for users
• Link to the relevant issue number/tracking issue for PEP 735 implementation in the PR description

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[finswimmer]

@sourcery-ai review

[finswimmer]

@sourcery-ai review

[sourcery-ai]

Hey @finswimmer - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider consolidating the two loops in _configure_package_dependencies that handle dependency groups (one for dependency-groups and one for tool.poetry.group) to reduce repeated logic.
• If a dependency group exists in the pyproject data but lacks a corresponding tool.poetry.group entry, consider whether you want to apply default optional settings explicitly.

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

Hey @finswimmer - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider consolidating the two loops in _configure_package_dependencies to reduce code duplication in processing dependency groups.
• Validate and normalize the dependency groups input early to ensure consistent types before iterating over them.

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟡 Testing: 1 issue found
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[radoering]

Support for https://peps.python.org/pep-0735/#dependency-object-specifiers is missing, which might be confusing for users.

[fosskers]

I'm sure there are many who are eagerly awaiting this, thank you.

[citelao]

Has development migrated to #837? Or is this PR blocked on that one? I'm happy to do some work to help this come in.

[sourcery-ai]

Hey @finswimmer - I've reviewed your changes - here's some feedback:

• The newly added dependency-groups-schema.json file appears empty; please populate it with the actual JSON schema so that validation of dependency-groups works correctly.
• _configure_package_dependencies is becoming quite large and does multiple passes over groups—consider extracting group initialization, constraint parsing, and include handling into smaller helper methods to improve readability and maintainability.
• _resolve_dependency_group_includes currently mutates the input dict in a loop for cycle detection; switching to a dedicated graph‐based algorithm or working on a copy could make the cycle detection logic clearer and side‐effect free.

Prompt for AI Agents

Please address the comments from this code review:
## Overall Comments
- The newly added dependency-groups-schema.json file appears empty; please populate it with the actual JSON schema so that validation of `dependency-groups` works correctly.
- _configure_package_dependencies is becoming quite large and does multiple passes over groups—consider extracting group initialization, constraint parsing, and include handling into smaller helper methods to improve readability and maintainability.
- _resolve_dependency_group_includes currently mutates the input dict in a loop for cycle detection; switching to a dedicated graph‐based algorithm or working on a copy could make the cycle detection logic clearer and side‐effect free.

## Individual Comments

### Comment 1
<location> `src/poetry/core/factory.py:453` </location>
<code_context>
+    ) -> dict[NormalizedName, list[NormalizedName]]:
+        resolved_groups: set[NormalizedName] = set()
+        included: dict[NormalizedName, list[NormalizedName]] = defaultdict(list)
+        while resolved_groups != set(dependency_groups):
+            for group, dependencies in dependency_groups.items():
+                if group in resolved_groups:
</code_context>

<issue_to_address>
Possible infinite loop in _resolve_dependency_group_includes if dependency_groups is malformed.

Consider adding a maximum iteration limit or an explicit error if the loop cannot make progress, to prevent potential infinite loops with malformed input.
</issue_to_address>

### Comment 2
<location> `src/poetry/core/factory.py:466` </location>
<code_context>
+                        resolved_dependencies.append(dep)
+                    else:
+                        included_group = canonicalize_name(dep["include-group"])
+                        if included_group in included[group]:
+                            raise ValueError(
+                                f"Cyclic dependency group include:"
</code_context>

<issue_to_address>
Cyclic dependency group detection may not catch all cycles.

The current logic only detects direct cycles. To handle indirect cycles, implement a more comprehensive cycle detection algorithm, such as depth-first search with a visited set.
</issue_to_address>

### Comment 3
<location> `src/poetry/core/factory.py:700` </location>
<code_context>
         ]
         result["errors"] += tool_poetry_validation_errors

+        dependency_groups = toml_data.get("dependency-groups")
+        if dependency_groups is not None:
+            dependency_groups_validation_errors = [
</code_context>

<issue_to_address>
Validation error messages for dependency-groups may be unclear.

Replacing 'data' with 'dependency-groups' in error messages may not always yield clear results, especially for complex schema errors. Please review the formatting to ensure messages remain accurate and user-friendly.
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
            dependency_groups_validation_errors = [
                e.replace("data", "dependency-groups")
                for e in validate_object(dependency_groups, "dependency-groups-schema")
            ]
=======
            dependency_groups_validation_errors = [
                (
                    f"dependency-groups: {e[5:]}" if e.startswith("data:") else
                    e.replace("data", "dependency-groups", 1) if e.startswith("data") else
                    e
                )
                for e in validate_object(dependency_groups, "dependency-groups-schema")
            ]
>>>>>>> REPLACE

</suggested_fix>

### Comment 4
<location> `src/poetry/core/packages/dependency.py:338` </location>
<code_context>

+    @classmethod
+    def _normalize_dependency_group_names(
+        cls,
+        dependency_groups: dict[str, list[str | dict[str, str]]],
+    ) -> dict[NormalizedName, list[str | dict[str, str]]]:
</code_context>

<issue_to_address>
The groups parameter in create_from_pep_508 is not documented.

Please update the docstring to include the groups parameter for clarity.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[radoering]

@sourcery-ai review

[sourcery-ai]

Hey @finswimmer - I've reviewed your changes - here's some feedback:

• The nested loops and branching in _configure_package_dependencies are getting quite large—consider splitting the PEP 735 vs legacy group resolution into smaller helper functions or classes to improve readability.
• _add_package_pep735_group_dependencies and _add_package_poetry_group_dependencies share similar logic—unifying or renaming them to better reflect their distinct responsibilities would reduce confusion.
• The duplicate‐name and include‐cycle checks in _validate_dependency_groups are intertwined and hard to follow; extracting them into separate validation helpers could simplify the overall logic.

Prompt for AI Agents

Please address the comments from this code review:
## Overall Comments
- The nested loops and branching in _configure_package_dependencies are getting quite large—consider splitting the PEP 735 vs legacy group resolution into smaller helper functions or classes to improve readability.
- _add_package_pep735_group_dependencies and _add_package_poetry_group_dependencies share similar logic—unifying or renaming them to better reflect their distinct responsibilities would reduce confusion.
- The duplicate‐name and include‐cycle checks in _validate_dependency_groups are intertwined and hard to follow; extracting them into separate validation helpers could simplify the overall logic.

## Individual Comments

### Comment 1
<location> `src/poetry/core/factory.py:98` </location>
<code_context>
+        dependencies: list[str | dict[str, str]],
+    ) -> list[str]:
+        group_includes = []
+        for constraint in dependencies:
+            if isinstance(constraint, str):
+                dep = Dependency.create_from_pep_508(
+                    constraint,
+                    relative_to=package.root_dir,
+                    groups=[group.pretty_name],
+                )
+                group.add_dependency(dep)
+            else:
+                group_includes.append(constraint["include-group"])
+        return group_includes
+
</code_context>

<issue_to_address>
Potential KeyError if 'include-group' is missing in dict constraint.

Use constraint.get("include-group") and handle the missing key to prevent KeyError exceptions.
</issue_to_address>

### Comment 2
<location> `src/poetry/core/factory.py:397` </location>
<code_context>
+            # with no corresponding entry in dependency-groups
+            # and add dependency information for existing groups
+            poetry_include_groups = {}
+            for group_name, group_config in tool_poetry_groups.items():
+                poetry_include_groups[group_name] = group_config.get(
+                    "include-groups", []
                 )
</code_context>

<issue_to_address>
Possible inconsistency in group name normalization between sources.

Ensure group names are normalized consistently across all sources to prevent mismatches due to case or separator differences.

Suggested implementation:

```python
            poetry_include_groups = {}
+            def _normalize_group_name(name):
+                # Normalize group names: lowercase and replace underscores with dashes
+                return name.lower().replace("_", "-")
+
+            for group_name, group_config in tool_poetry_groups.items():
+                normalized_group_name = _normalize_group_name(group_name)
+                poetry_include_groups[normalized_group_name] = group_config.get(
+                    "include-groups", []
+                )

```

If group names are used elsewhere in this function or file (e.g., when reading from or writing to `poetry_include_groups`), ensure those accesses also use the `_normalize_group_name()` function for consistency.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[radoering]

I refactored the logic for group includes a bit so that we use the same logic for PEP 735 groups and tool.poetry groups.

The PR should be ready for merging now but I think I will wait until I reviewed python-poetry/poetry#10130, in case I spot any issues while reviewing this one.