-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Avoid pip install --dry-run
downloading full wheels
#13482
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
req_set.add_named_requirement(ireq) | ||
|
||
reqs = req_set.all_requirements | ||
self.factory.preparer.prepare_linked_requirements_more(reqs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the key change - the resolver no longer triggers additional preparation. Instead, we move this responsibility to consumers of the resolve result.
Technically, I could have made the preparer
aware of the fact that we don't want to download anything, but the preparer is already a grab-bag of flags, and it felt cleaner to do it this way.
Thanks for opening the pull request with pip, I'm certainly happy to see this PR, but please be aware it might take a moment for a pip maintainer to review this, as we all do this on a volunteer basis. |
* Removed "more preparation" (downloading) from the resolver to prevent downloading before dry-run validation * Added distribution caching to `InstallRequirement` with `set_dist()` and `get_dist()` methods to preserve metadata-only distributions * Set `download_info` during metadata-only fetching to ensure it's available for commands like `pip lock` and `--report` without requiring full downloads Closes pypa#12603.
ff4715b
to
196f992
Compare
Thanks again for raising this PR, I've put it on my list of things to review, though it may be a few weeks before I have enough time to give full feedback. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall I am very happy with this approach, the news item needs updating, and I am going to merge in the latest main to check no changes elsewhere have caused anything to fail.
It would be nice to have at least one integration test, but if you can not provide this I will add one in a follow up PR.
news/12603.feature.rst
Outdated
@@ -0,0 +1 @@ | |||
When PEP-658 metadata is available, full distribution download no longer occurs when using dry-run mode on install. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to be updated slightly, in-particular all sdists are still downloaded (at least for now), either specifically call out wheels or change to something like:
When PEP-658 metadata is available for a distribution the full file is no longer downloaded
Or
When PEP-658 metadata is available, full wheel download no longer occurs
Further, also add that pip lock
no longer downloads distributions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there's no update and no objections in the next few days I will update this news items myself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Technically pip
has no problem fetching metadata for sdists already 😉. So this PR does make --dry-run
avoid an sdist download/build 🎉 (this is a pretty substantial benefit as it means you don't need to build the sdist to figure out the dependencies... if only we had any repositories with sdist metadata served ala 658).
To prove this, I did the following:
mkdir -p ./delme3-project; cd ./delme3-project
cat << EOF > pyproject.toml
[project]
name = "delme3"
version = "0.1.0"
requires-python = ">=3.11"
EOF
python -m venv ./venv
source ./venv/bin/activate
python -m pip install build
python -m build --sdist .
mkdir -p dummy-repo/delme3
mv dist/delme3-0.1.0.tar.gz ./dummy-repo/delme3/
tar -xOf ./dummy-repo/delme3/delme3-0.1.0.tar.gz delme3-0.1.0/PKG-INFO > ./dummy-repo/delme3/delme3-0.1.0.tar.gz.metadata
cat << EOF > ./dummy-repo/delme3/index.html
<!DOCTYPE html><html><body>
<a href="/delme3/delme3-0.1.0.tar.gz" data-core-metadata="true">delme3-0.1.0.tar.gz</a><br/>
</body></html>
EOF
cd dummy-repo
python -m http.server
On the pip side, I ran:
$ python -m pip install delme3 -v --no-cache --dry-run --index-url http://localhost:8000/
Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11)
Looking in indexes: http://localhost:8000/
Collecting delme3
Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata
Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes)
Would install delme3-0.1.0
With only the following http.server
output:
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [24/Sep/2025 12:43:06] "GET /delme3/ HTTP/1.1" 200 -
127.0.0.1 - - [24/Sep/2025 12:43:06] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 -
(i.e. the sdist is never downloaded)
Whereas on master (f2b9231):
$ python -m pip install delme3 -v --no-cache --dry-run --index-url http://localhost:8000/
Using pip 25.3.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11)
Looking in indexes: http://localhost:8000/
Collecting delme3
Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata
Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes)
Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz (646 bytes)
Running command pip subprocess to install build dependencies
Using pip 25.3.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11)
Looking in indexes: http://localhost:8000/
ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
ERROR: No matching distribution found for setuptools>=40.8.0
Ensuring compatibility with Acc-Py
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /media/important/github/pypa/pip/venv/bin/python /media/important/github/pypa/pip/src/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-ohiuyo5e/overlay --no-warn-script-location --disable-pip-version-check --no-compile --target '' -v --no-binary :none: --only-binary :none: -i http://localhost:8000/ --trusted-host acc-py-repo.cern.ch -- 'setuptools>=40.8.0'
cwd: [inherit]
Installing build dependencies ... error
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
With the http log:
$ python -m http.server
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/ HTTP/1.1" 200 -
127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 -
127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/delme3-0.1.0.tar.gz HTTP/1.1" 200 -
127.0.0.1 - - [24/Sep/2025 12:43:56] code 404, message File not found
127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /setuptools/ HTTP/1.1" 404 -
I proceeded to try pip lock
out on the above setup, but got a failure:
$ python -m pip lock delme3 -v --no-cache --index-url http://localhost:8000/
Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11)
WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning.
Looking in indexes: http://localhost:8000/
Collecting delme3
Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata
Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes)
ERROR: Exception:
Traceback (most recent call last):
File "/media/important/github/pypa/pip/src/pip/_internal/cli/base_command.py", line 107, in _run_wrapper
status = _inner_run()
^^^^^^^^^^^^
File "/media/important/github/pypa/pip/src/pip/_internal/cli/base_command.py", line 98, in _inner_run
return self.run(options, args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/media/important/github/pypa/pip/src/pip/_internal/cli/req_command.py", line 70, in wrapper
return func(self, options, args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/important/github/pypa/pip/src/pip/_internal/commands/lock.py", line 162, in run
pylock_toml = Pylock.from_install_requirements(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 179, in from_install_requirements
packages=sorted(
^^^^^^^
File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 181, in <genexpr>
Package.from_install_requirement(ireq, base_dir)
File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 138, in from_install_requirement
raise NotImplementedError()
NotImplementedError
I think that is because my package repository didn't include hashes...
This was the changed line in question:
$ cat ./delme3/index.html | grep sha256
<a href="/delme3/delme3-0.1.0.tar.gz#sha256=713524ababddfe0944ccc8bef33535a13e5ddcb92156dddfc37fc838c8e91acc" data-core-metadata="true">delme3-0.1.0.tar.gz foo</a><br/>
When I added them it correctly generated the lock file without downloading:
$ python -m pip lock delme3 -v --no-cache --index-url http://localhost:8000/
Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11)
WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning.
Looking in indexes: http://localhost:8000/
Collecting delme3
Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata
Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes)
$ python -m http.server
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [24/Sep/2025 12:56:44] "GET /delme3/ HTTP/1.1" 200 -
127.0.0.1 - - [24/Sep/2025 12:56:44] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 -
🎉
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you like to mention anything about performance in the what's new? Whilst pip
doesn't quite reach uv
performance, it is still very reasonable when the cache is cold! The narrative that pip
is much slower might be nice to counter here 😉
$ time uv pip install --dry-run torch --no-cache --index-url https://pypi.org/simple/
Using Python 3.11.11 environment at: venv
Resolved 25 packages in 371ms
Would download 24 packages
Would install 24 packages
+ filelock==3.19.1
+ fsspec==2025.9.0
+ jinja2==3.1.6
+ markupsafe==3.0.2
+ mpmath==1.3.0
+ networkx==3.5
+ nvidia-cublas-cu12==12.8.4.1
+ nvidia-cuda-cupti-cu12==12.8.90
+ nvidia-cuda-nvrtc-cu12==12.8.93
+ nvidia-cuda-runtime-cu12==12.8.90
+ nvidia-cudnn-cu12==9.10.2.21
+ nvidia-cufft-cu12==11.3.3.83
+ nvidia-cufile-cu12==1.13.1.3
+ nvidia-curand-cu12==10.3.9.90
+ nvidia-cusolver-cu12==11.7.3.90
+ nvidia-cusparse-cu12==12.5.8.93
+ nvidia-cusparselt-cu12==0.7.1
+ nvidia-nccl-cu12==2.27.3
+ nvidia-nvjitlink-cu12==12.8.93
+ nvidia-nvtx-cu12==12.8.90
+ sympy==1.14.0
+ torch==2.8.0
+ triton==3.4.0
+ typing-extensions==4.15.0
real 0m0.492s
user 0m0.132s
sys 0m0.039s
$ time python -m pip lock torch --no-cache --index-url https://pypi.org/simple/
WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning.
Looking in indexes: https://pypi.org/simple/
Collecting torch
Downloading torch-2.8.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (30 kB)
Collecting filelock (from torch)
Downloading filelock-3.19.1-py3-none-any.whl.metadata (2.1 kB)
Collecting typing-extensions>=4.10.0 (from torch)
Downloading typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB)
Collecting sympy>=1.13.3 (from torch)
Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
Collecting networkx (from torch)
Downloading networkx-3.5-py3-none-any.whl.metadata (6.3 kB)
Collecting jinja2 (from torch)
Downloading jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
Collecting fsspec (from torch)
Downloading fsspec-2025.9.0-py3-none-any.whl.metadata (10 kB)
Collecting nvidia-cuda-nvrtc-cu12==12.8.93 (from torch)
Downloading nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cuda-runtime-cu12==12.8.90 (from torch)
Downloading nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cuda-cupti-cu12==12.8.90 (from torch)
Downloading nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cudnn-cu12==9.10.2.21 (from torch)
Downloading nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-cublas-cu12==12.8.4.1 (from torch)
Downloading nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cufft-cu12==11.3.3.83 (from torch)
Downloading nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-curand-cu12==10.3.9.90 (from torch)
Downloading nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cusolver-cu12==11.7.3.90 (from torch)
Downloading nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-cusparse-cu12==12.5.8.93 (from torch)
Downloading nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-cusparselt-cu12==0.7.1 (from torch)
Downloading nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (7.0 kB)
Collecting nvidia-nccl-cu12==2.27.3 (from torch)
Downloading nvidia_nccl_cu12-2.27.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.0 kB)
Collecting nvidia-nvtx-cu12==12.8.90 (from torch)
Downloading nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-nvjitlink-cu12==12.8.93 (from torch)
Downloading nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cufile-cu12==1.13.1.3 (from torch)
Downloading nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting triton==3.4.0 (from torch)
Downloading triton-3.4.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.7 kB)
Collecting setuptools>=40.8.0 (from triton==3.4.0->torch)
Downloading setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB)
Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch)
Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Collecting MarkupSafe>=2.0 (from jinja2->torch)
Downloading MarkupSafe-3.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
real 0m1.555s
user 0m0.807s
sys 0m0.104s
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting about the sdists, I wasn't aware of that impact, I'll try and review later this week.
The Windows test failure looks unrelated. I'll try and make, or review, a PR to fix as soon as I can.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the review! News updated
news/12603.feature.rst
Outdated
@@ -0,0 +1 @@ | |||
When PEP-658 metadata is available, full distribution download no longer occurs when using dry-run mode on install. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Technically pip
has no problem fetching metadata for sdists already 😉. So this PR does make --dry-run
avoid an sdist download/build 🎉 (this is a pretty substantial benefit as it means you don't need to build the sdist to figure out the dependencies... if only we had any repositories with sdist metadata served ala 658).
To prove this, I did the following:
mkdir -p ./delme3-project; cd ./delme3-project
cat << EOF > pyproject.toml
[project]
name = "delme3"
version = "0.1.0"
requires-python = ">=3.11"
EOF
python -m venv ./venv
source ./venv/bin/activate
python -m pip install build
python -m build --sdist .
mkdir -p dummy-repo/delme3
mv dist/delme3-0.1.0.tar.gz ./dummy-repo/delme3/
tar -xOf ./dummy-repo/delme3/delme3-0.1.0.tar.gz delme3-0.1.0/PKG-INFO > ./dummy-repo/delme3/delme3-0.1.0.tar.gz.metadata
cat << EOF > ./dummy-repo/delme3/index.html
<!DOCTYPE html><html><body>
<a href="/delme3/delme3-0.1.0.tar.gz" data-core-metadata="true">delme3-0.1.0.tar.gz</a><br/>
</body></html>
EOF
cd dummy-repo
python -m http.server
On the pip side, I ran:
$ python -m pip install delme3 -v --no-cache --dry-run --index-url http://localhost:8000/
Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11)
Looking in indexes: http://localhost:8000/
Collecting delme3
Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata
Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes)
Would install delme3-0.1.0
With only the following http.server
output:
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [24/Sep/2025 12:43:06] "GET /delme3/ HTTP/1.1" 200 -
127.0.0.1 - - [24/Sep/2025 12:43:06] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 -
(i.e. the sdist is never downloaded)
Whereas on master (f2b9231):
$ python -m pip install delme3 -v --no-cache --dry-run --index-url http://localhost:8000/
Using pip 25.3.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11)
Looking in indexes: http://localhost:8000/
Collecting delme3
Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata
Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes)
Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz (646 bytes)
Running command pip subprocess to install build dependencies
Using pip 25.3.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11)
Looking in indexes: http://localhost:8000/
ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
ERROR: No matching distribution found for setuptools>=40.8.0
Ensuring compatibility with Acc-Py
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /media/important/github/pypa/pip/venv/bin/python /media/important/github/pypa/pip/src/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-ohiuyo5e/overlay --no-warn-script-location --disable-pip-version-check --no-compile --target '' -v --no-binary :none: --only-binary :none: -i http://localhost:8000/ --trusted-host acc-py-repo.cern.ch -- 'setuptools>=40.8.0'
cwd: [inherit]
Installing build dependencies ... error
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
With the http log:
$ python -m http.server
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/ HTTP/1.1" 200 -
127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 -
127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/delme3-0.1.0.tar.gz HTTP/1.1" 200 -
127.0.0.1 - - [24/Sep/2025 12:43:56] code 404, message File not found
127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /setuptools/ HTTP/1.1" 404 -
I proceeded to try pip lock
out on the above setup, but got a failure:
$ python -m pip lock delme3 -v --no-cache --index-url http://localhost:8000/
Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11)
WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning.
Looking in indexes: http://localhost:8000/
Collecting delme3
Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata
Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes)
ERROR: Exception:
Traceback (most recent call last):
File "/media/important/github/pypa/pip/src/pip/_internal/cli/base_command.py", line 107, in _run_wrapper
status = _inner_run()
^^^^^^^^^^^^
File "/media/important/github/pypa/pip/src/pip/_internal/cli/base_command.py", line 98, in _inner_run
return self.run(options, args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/media/important/github/pypa/pip/src/pip/_internal/cli/req_command.py", line 70, in wrapper
return func(self, options, args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/important/github/pypa/pip/src/pip/_internal/commands/lock.py", line 162, in run
pylock_toml = Pylock.from_install_requirements(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 179, in from_install_requirements
packages=sorted(
^^^^^^^
File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 181, in <genexpr>
Package.from_install_requirement(ireq, base_dir)
File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 138, in from_install_requirement
raise NotImplementedError()
NotImplementedError
I think that is because my package repository didn't include hashes...
This was the changed line in question:
$ cat ./delme3/index.html | grep sha256
<a href="/delme3/delme3-0.1.0.tar.gz#sha256=713524ababddfe0944ccc8bef33535a13e5ddcb92156dddfc37fc838c8e91acc" data-core-metadata="true">delme3-0.1.0.tar.gz foo</a><br/>
When I added them it correctly generated the lock file without downloading:
$ python -m pip lock delme3 -v --no-cache --index-url http://localhost:8000/
Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11)
WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning.
Looking in indexes: http://localhost:8000/
Collecting delme3
Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata
Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes)
$ python -m http.server
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [24/Sep/2025 12:56:44] "GET /delme3/ HTTP/1.1" 200 -
127.0.0.1 - - [24/Sep/2025 12:56:44] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 -
🎉
news/12603.feature.rst
Outdated
@@ -0,0 +1 @@ | |||
When PEP-658 metadata is available, full distribution download no longer occurs when using dry-run mode on install. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you like to mention anything about performance in the what's new? Whilst pip
doesn't quite reach uv
performance, it is still very reasonable when the cache is cold! The narrative that pip
is much slower might be nice to counter here 😉
$ time uv pip install --dry-run torch --no-cache --index-url https://pypi.org/simple/
Using Python 3.11.11 environment at: venv
Resolved 25 packages in 371ms
Would download 24 packages
Would install 24 packages
+ filelock==3.19.1
+ fsspec==2025.9.0
+ jinja2==3.1.6
+ markupsafe==3.0.2
+ mpmath==1.3.0
+ networkx==3.5
+ nvidia-cublas-cu12==12.8.4.1
+ nvidia-cuda-cupti-cu12==12.8.90
+ nvidia-cuda-nvrtc-cu12==12.8.93
+ nvidia-cuda-runtime-cu12==12.8.90
+ nvidia-cudnn-cu12==9.10.2.21
+ nvidia-cufft-cu12==11.3.3.83
+ nvidia-cufile-cu12==1.13.1.3
+ nvidia-curand-cu12==10.3.9.90
+ nvidia-cusolver-cu12==11.7.3.90
+ nvidia-cusparse-cu12==12.5.8.93
+ nvidia-cusparselt-cu12==0.7.1
+ nvidia-nccl-cu12==2.27.3
+ nvidia-nvjitlink-cu12==12.8.93
+ nvidia-nvtx-cu12==12.8.90
+ sympy==1.14.0
+ torch==2.8.0
+ triton==3.4.0
+ typing-extensions==4.15.0
real 0m0.492s
user 0m0.132s
sys 0m0.039s
$ time python -m pip lock torch --no-cache --index-url https://pypi.org/simple/
WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning.
Looking in indexes: https://pypi.org/simple/
Collecting torch
Downloading torch-2.8.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (30 kB)
Collecting filelock (from torch)
Downloading filelock-3.19.1-py3-none-any.whl.metadata (2.1 kB)
Collecting typing-extensions>=4.10.0 (from torch)
Downloading typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB)
Collecting sympy>=1.13.3 (from torch)
Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
Collecting networkx (from torch)
Downloading networkx-3.5-py3-none-any.whl.metadata (6.3 kB)
Collecting jinja2 (from torch)
Downloading jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
Collecting fsspec (from torch)
Downloading fsspec-2025.9.0-py3-none-any.whl.metadata (10 kB)
Collecting nvidia-cuda-nvrtc-cu12==12.8.93 (from torch)
Downloading nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cuda-runtime-cu12==12.8.90 (from torch)
Downloading nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cuda-cupti-cu12==12.8.90 (from torch)
Downloading nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cudnn-cu12==9.10.2.21 (from torch)
Downloading nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-cublas-cu12==12.8.4.1 (from torch)
Downloading nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cufft-cu12==11.3.3.83 (from torch)
Downloading nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-curand-cu12==10.3.9.90 (from torch)
Downloading nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cusolver-cu12==11.7.3.90 (from torch)
Downloading nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-cusparse-cu12==12.5.8.93 (from torch)
Downloading nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-cusparselt-cu12==0.7.1 (from torch)
Downloading nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (7.0 kB)
Collecting nvidia-nccl-cu12==2.27.3 (from torch)
Downloading nvidia_nccl_cu12-2.27.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.0 kB)
Collecting nvidia-nvtx-cu12==12.8.90 (from torch)
Downloading nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-nvjitlink-cu12==12.8.93 (from torch)
Downloading nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cufile-cu12==1.13.1.3 (from torch)
Downloading nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting triton==3.4.0 (from torch)
Downloading triton-3.4.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.7 kB)
Collecting setuptools>=40.8.0 (from triton==3.4.0->torch)
Downloading setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB)
Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch)
Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Collecting MarkupSafe>=2.0 (from jinja2->torch)
Downloading MarkupSafe-3.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
real 0m1.555s
user 0m0.807s
sys 0m0.104s
Closes #12603.
InstallRequirement
withset_dist()
andget_dist()
methods to preserve metadata-only distributionsdownload_info
during metadata-only fetching to ensure it's available for commands likepip lock
and--report
without requiring full downloads