Misleading `ResolutionImpossible` message w/ a requirement duplicate and an incompatible Python runtime

[webknjaz]

Description

I've faced this by updating a pip-tools-managed constraints file under Python 3.9 but using these pins under Python 3.6. Obviously, some of them were incompatible (which I didn't notice immediately) but the problem I'm reporting is how pip communicates what's happened.

Pip errors out, claiming that two user requests of the same package are conflicting, when they are not. One of the requests has an unspecified requested version so basically anything matches, and the other one is pinned to a specific release. So basically out of these two, the resolver would select the pinned one, and it'd be compatible with a more broad request.

Keeping the above in mind, what pip suggests in the error message is not really true/obvious. I suppose that because the pin has incompatible python_version, it's facing an actual (invisible to the user) conflict. But this information isn't mentioned in the log which leads to a misleading impression.

I've crafted a reproducer below.

Expected behavior

The error message should specify that it's Require-Python that causes the problem, not two user requests that are actually compatible with each other.

pip version

Unknown but I reproduced this locally with v22.2.1 and v21.3.1

Python version

3.6 on CI, reproduced locally with 3.8

OS

macos-latest @ GHA, Gentoo Linux locally

How to Reproduce

$ python3.7 -m pip install django django==4.0.6
Collecting django
  Using cached Django-3.2.14-py3-none-any.whl (7.9 MB)
ERROR: Cannot install django and django==4.0.6 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested django
    The user requested django==4.0.6

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

(in my case, this was caused by setuptools with older pip and older Python version but I've found another package to demonstrate it under a supported Python version)

Output

ERROR: Cannot install setuptools because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested setuptools
    The user requested (constraint) setuptools==63.2.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[webknjaz]

This is potentially similar to #9811 and #10953. I haven't attempted to debug this, though. Just made an educated guess based on the behavior.

[uranusjr]

#10421 maye help with this.

[webknjaz]

Oh, I didn't realize @pradyunsg is still doing that effort.

[notatallshaw]

I'm working on a couple of improvements to this, first is #13588

The second is to highlight when requirements have no valid distributions on your platform, with the idea to do something like this:

Collecting django
  Using cached Django-3.2.14-py3-none-any.whl (7.9 MB)
ERROR: Cannot install django and django==4.0.6 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested django
    The user requested django==4.0.6 [*]

[*]: There are no matching distributions available for the starred requirements, use "pip index versions" to see what versions have matching distributions for your environment.

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

This is achievable without needing to extract any additional information resolvelib, it wouldn't fix all situations but I think would capture a big majority of them.

resolvelib itself is not designed with explainability in mind, I think other approaches are going to be tricky unless we replace resolvelib.