Skip to content

Conversation

@dhellmann
Copy link
Member

@dhellmann dhellmann commented Sep 6, 2025

Add a new --reduce flag to the graph to-dot command that filters the
dependency graph to only include nodes with fromager customizations
(overrides, settings, patches, plugins, or pre-built wheels).

The reduced graph preserves dependency relationships by creating direct
edges between customized nodes, skipping intermediate nodes without
customizations. Original requirement specifications are tracked and
displayed in edge labels for better clarity.

Key changes:

  • Add --reduce command line option to to-dot command
  • Implement has_customizations() to identify nodes with customizations
  • Implement reduce_graph() to build filtered dependency relationships
  • Track original requirement strings for proper edge labeling
  • Use req_type.is_build_requirement property for build dependency checks

This feature helps visualize which packages in a dependency tree actually
have fromager customizations while maintaining essential dependency
relationships.

Fixes #715

Cursor chat transcript: https://gist.github.com/dhellmann/e362954e00a7270dcc71037382f53853

Add a new --reduce flag to the graph to-dot command that filters the
dependency graph to only include nodes with fromager customizations
(overrides, settings, patches, plugins, or pre-built wheels).

The reduced graph preserves dependency relationships by creating direct
edges between customized nodes, skipping intermediate nodes without
customizations. Original requirement specifications are tracked and
displayed in edge labels for better clarity.

Key changes:
- Add --reduce command line option to to-dot command
- Implement has_customizations() to identify nodes with customizations
- Implement reduce_graph() to build filtered dependency relationships
- Track original requirement strings for proper edge labeling
- Use req_type.is_build_requirement property for build dependency checks

This feature helps visualize which packages in a dependency tree actually
have fromager customizations while maintaining essential dependency
relationships.

Cursor chat transcript: https://gist.github.com/dhellmann/e362954e00a7270dcc71037382f53853

Co-authored-by: Claude 3.5 Sonnet (Anthropic AI Assistant)
Signed-off-by: Doug Hellmann <[email protected]>
@dhellmann dhellmann requested a review from a team as a code owner September 6, 2025 21:54
@dhellmann
Copy link
Member Author

Here are a few sample output files showing the difference between a full graph and a reduced graph:

docling-graph.json.dot.pdf
docling-graph.json-reduced.dot.pdf
torch-cuda-graph.json.dot.pdf
torch-cuda-graph.json-reduced.dot.pdf

@LalatenduMohanty
Copy link
Member

@dhellmann can you pass a graph file which has packages with customization, so that it would be easier for me to test the PR?

Rename the --reduce command line option to --overrides-only to better
reflect its purpose of filtering the dependency graph to show only nodes
with fromager overrides.

Update the help text to clarify that it includes nodes with settings,
patches, or plugins, removing the redundant mention of "customizations"
and "overrides" in the same description.

Co-authored-by: Claude 3.5 Sonnet (Anthropic AI Assistant)
Signed-off-by: Doug Hellmann <[email protected]>
@dhellmann
Copy link
Member Author

@dhellmann can you pass a graph file which has packages with customization, so that it would be easier for me to test the PR?

The graph file itself doesn't record which packages had customizations, though we could add that. I'll upload the graph files I used to produce the PDFs but you'll have to run fromager with the context from our downstream configuration repo to get the same output.

docling-graph.json

torch-cuda-graph.json

# If the child doesn't have customizations, continue traversing
stack.append((child, new_path, current_req))

return customized_nodes, reduced_dependencies
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will suggest that we break the reduce_graph() to smaller methods. Currently it is doing few things. Smaller functions will improve its readability.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would you break it up?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dhellmann IMO we should merge the PR as it is and we can take up refactoring in a separate issue. I would be happy to work on it.

Copy link
Member

@LalatenduMohanty LalatenduMohanty Oct 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the idea I had (using Cursor), I have the code locally as well.

reduce_graph() function can be broken down into 4 focused helper functions:

  1. _get_nodes_for_reduction() (9 lines)
    Purpose: Determine starting node set based on install_only flag
    Responsibility: Node selection logic

  2. _find_customized_nodes() (10 lines)
    Purpose: Filter nodes to find only those with customizations
    Responsibility: Node filtering logic

  3. _find_customized_dependencies_for_node() (33 lines)
    Purpose: Complex graph traversal for a single node
    Responsibility: Depth-first search algorithm
    Most complex: Contains the core traversal logic

  4. _build_reduced_dependency_map() (8 lines)
    Purpose: Orchestrate dependency mapping for all nodes
    Responsibility: Coordinate traversal across all customized nodes

  5. reduce_graph() (Refactored, 13 lines)
    Clear flow: Get nodes → Find customized → Build dependencies → Return

Simplify the logic for reducing a graph by moving the check into the
PackageBuildInfo class.
Copy link
Contributor

@rd4398 rd4398 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good! I see there are few comments on MR so I will wait for them to be resolved before I hit approve

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

simplified graph mode

4 participants