Skip to content

Conversation

@basnijholt
Copy link
Owner

@basnijholt basnijholt commented Aug 28, 2025

Summary

  • Fixed a bug where local dependencies referenced by multiple projects were duplicated in the pip install command
  • Added comprehensive test coverage for the deduplication logic
  • The issue was causing packages to appear multiple times (e.g., 9 times) in CI builds

Problem

When multiple projects reference the same local dependency through local_dependencies: in their requirements.yaml, the dependency was being added multiple times to the pip install command. This was observed in CI where third_party/cypress appeared 9 times in a single pip install command.

Root Cause

The deduplication logic in _install_command function was flawed:

  • It created installable_set from existing packages
  • But didn't update this set while iterating through local_dependencies.values()
  • If the same dependency appeared in multiple projects' local_dependencies, it would be added multiple times

Solution

Updated the deduplication logic to maintain the installable_set as dependencies are added:

# Now properly updates the set to prevent duplicates
for deps in local_dependencies.values():
    for dep in deps:
        dep_resolved = dep.resolve()
        if dep_resolved not in installable_set:
            installable.append(dep)
            installable_set.add(dep_resolved)  # Key fix

Test Plan

  • Added test test_no_duplicate_local_dependencies_in_install_command that reproduces the issue
  • Test fails without the fix (showing 3 duplicates)
  • Test passes with the fix (showing proper deduplication)
  • Existing tests continue to pass

Related Issues


📚 Documentation preview 📚: https://unidep--255.org.readthedocs.build/en/255/

basnijholt and others added 3 commits August 28, 2025 12:04
This test demonstrates a bug where local dependencies that are
referenced by multiple projects appear multiple times in the
pip install command. The test currently fails, showing that the
same dependency is added 3 times instead of once.

Related to CI issue: https://github.com/ionq/system_performance/actions/runs/17304244901/job/49123013838?pr=925
When multiple projects reference the same local dependency, the
deduplication logic was not properly preventing duplicates. The
installable_set was created once but not updated as new dependencies
were added, causing the same dependency to appear multiple times.

This fix updates the installable_set each time a new dependency is
added, ensuring proper deduplication.

Fixes the issue where packages like third_party/cypress were being
repeated 9 times in the pip install command in CI.
@codecov
Copy link

codecov bot commented Aug 28, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.13%. Comparing base (06be4a8) to head (09dfdc8).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #255   +/-   ##
=======================================
  Coverage   97.12%   97.13%           
=======================================
  Files          10       10           
  Lines        1497     1502    +5     
=======================================
+ Hits         1454     1459    +5     
  Misses         43       43           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants