ci: migrate from pip to uv for faster installs#1431
Conversation
- Replace all pip install calls with uv pip install --system - Add astral-sh/setup-uv@v5 step to all workflows - Remove unnecessary pip upgrade steps - Use uv build instead of python -m build in pypi_upload Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The code_quality jobs didn't have setup-python, so they used the system Python on ubuntu-latest which has PEP 668 restrictions that block --system installs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1431 +/- ##
===========================================
+ Coverage 42.56% 60.34% +17.77%
===========================================
Files 62 62
Lines 5248 5248
===========================================
+ Hits 2234 3167 +933
+ Misses 3014 2081 -933 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Measured speedups from PR gate runComparing the latest PR gate run (uv) against the most recent main run (pip): CPU integration tests
Unit tests — Windows (biggest absolute wins)
Unit tests — Linux
Unit tests — macOS
All 17 jobs passed ✅ |
ci_linux resultsCompared the ci/uv-install run against the latest main run. 37/38 jobs passed (one pre-existing ONNX segfault on Python 3.10, unrelated). Per-job install speedupsCPU tests: install time dropped from ~175–228s to 73–91s (2.0–2.7×) Wall-time impactSince these jobs are parallelized, what matters is the critical path:
The biggest impact is on the PR gate workflow, where install time is a larger fraction of total job time — especially on Windows (up to 5× faster installs, see previous comment). |
.github/workflows/code_quality.yml
Outdated
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.12" |
There was a problem hiding this comment.
If this is shared amongst all builds, perhaps make it a variable?
Address review feedback: use a single PYTHON_VERSION env var instead of repeating '3.12' in each job. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Motivation
Dependency installation is a significant bottleneck in our CI pipelines. Looking at recent runs of the
pull_requestworkflow:uv is a drop-in replacement for pip that's 10–100× faster at dependency resolution and installation. Conservatively, this should cut install times by 5–10×, saving minutes per CI run — especially on the heavier workflows (cpu_tests, gpu_tests, notebooks).
Changes
Replace
pip installwithuv pip install --systemacross all 8 CI workflows that install Python packages:astral-sh/setup-uv@v5step to all jobspip install/python -m pip install→uv pip install --systempip install --upgrade pipstepsuv buildinstead ofpython -m pip install build && python -m buildin pypi_uploadNo changes to
pyproject.toml, the build backend (still setuptools), or test configuration. The--systemflag tells uv to install into the system Python rather than creating a venv, matching the existing CI behavior.Affected workflows
call_cpu_testscall_gpu_testspull_request(unit_tests, Windows)pull_request(unit_tests, Linux/macOS)code_qualityci_docsci_credentialsnotebook_testspypi_uploadWhat this doesn't do (yet)
This PR only swaps the installer. A natural follow-up would be adding
uv lockto generate auv.lockfile for fully reproducible CI environments with pinned transitive dependencies.