Skip to content

Conglomerated changes to optimize venv usage in pipeline #42486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

scbedd
Copy link
Member

@scbedd scbedd commented Aug 12, 2025

todo:

  • Update how we reference the virtual env for the pipeline according to the learnings from my test build
  • Swap to uv pip instead of python -m pip in core usage, with detection and fallback to python -m pip install where necessary.
  • Optimize all python -m pip calls from any CI to reference a $(PIP_EXE) that is set.
    • explore using a symlink for pip instead of actually referencing the target exe directly or by environment variable

@Copilot Copilot AI review requested due to automatic review settings August 12, 2025 19:29
@scbedd scbedd self-assigned this Aug 12, 2025
@scbedd scbedd moved this to 🔬 Dev in PR in Azure SDK EngSys 🍕 Aug 12, 2025
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request optimizes virtual environment usage in Azure DevOps pipelines by introducing UV (a fast Python package installer) as the preferred tool while maintaining fallbacks to traditional pip commands. The changes streamline environment setup and package installation across the build pipeline.

Key changes:

  • Integration of UV package manager with automatic fallback to pip when UV is unavailable
  • Simplified virtual environment activation by removing manual activation scripts in favor of environment variables
  • Addition of a new pipeline template for UV installation across different operating systems

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
eng/scripts/create-venv.ps1 Added UV detection and usage for virtual environment creation with fallback to virtualenv
eng/scripts/activate-venv.ps1 Simplified activation by setting VIRTUAL_ENV and PATH variables instead of sourcing activation scripts
eng/scripts/Language-Settings.ps1 Added UV pip usage for package installations with fallback logic
eng/pipelines/templates/steps/use-venv.yml Integrated UV installation template into virtual environment setup
eng/pipelines/templates/steps/seed-virtualenv-wheels.yml Replaced pip with UV for virtualenv installation
eng/pipelines/templates/steps/release-candidate-steps.yml Removed manual activation scripts and switched to UV for dependency installation
eng/pipelines/templates/steps/install-uv.yml New template for cross-platform UV installation
eng/pipelines/templates/steps/build-test.yml Simplified by removing activation scripts and using UV for package installation
eng/pipelines/templates/steps/build-package-artifacts.yml Updated to use UV and removed activation script dependencies
eng/pipelines/templates/steps/build-extended-artifacts.yml Switched from pip to UV for dependency installation
eng/pipelines/templates/steps/analyze.yml Replaced pip with UV for tool installation

@@ -5,7 +5,7 @@ parameters:

steps:
- pwsh: |
python -m pip install virtualenv
uv pip install virtualenv
Copy link
Preview

Copilot AI Aug 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step should include error handling to fall back to python -m pip install virtualenv if UV is not available, similar to the pattern used in other files. Without this fallback, the step will fail on systems where UV installation failed or is not available.

Suggested change
uv pip install virtualenv
uv pip install virtualenv
if ($LASTEXITCODE -ne 0) {
Write-Host "uv not available, falling back to python -m pip install virtualenv"
python -m pip install virtualenv
}

Copilot uses AI. Check for mistakes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're probably right here. Will make an adjustment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🔬 Dev in PR
Development

Successfully merging this pull request may close these issues.

1 participant