build(workflows): add pip-audit CI dependency scanning (#1052)

WilliamBerryiii · Bill Berry · web-flow · commit c77e064f9527 · 2026-03-16T19:16:54.000-07:00
## Description Added Python dependency vulnerability scanning through *pip-audit* at both CI and local execution levels. A new **reusable GitHub Actions workflow** audits Python projects via `uvx pip-audit@2.10.0`, with change-detection optimization that skips audits when no dependency files changed. The workflow integrates into PR validation through a matrix job gated on discovered Python projects. The existing **dependency-review workflow** gained a license allowlist of 13 OSI-approved SPDX identifiers and OSSF Scorecard display with a warning threshold. A **local PowerShell wrapper** (*Invoke-PipAudit.ps1*) mirrors the CI scanning behavior and exposes an `audit:pip` npm script for developer use. All new functionality includes **11 Pester tests** covering discovery, audit execution, and orchestration paths. ### CI Pipeline > The reusable workflow bridges the gap between *uv*-managed lock files and *pip-audit*'s requirements.txt input using `uv export`. - Added *pip-audit.yml* reusable workflow with `working-directory`, `soft-fail`, and `changed-files-only` inputs - Implemented dependency-file change detection via `git diff` scoped to *pyproject.toml*, *uv.lock*, and *requirements\*.txt* - Pinned *pip-audit* to version 2.10.0, all actions SHA-pinned with version comments - *actions/checkout* v4.2.2, *astral-sh/setup-uv* v7.5.0, *actions/setup-python* v6.2.0, *actions/upload-artifact* v4.4.3 - Uploaded JSON audit results as artifacts with 30-day retention - Set `permissions: contents: read` at workflow and job level ### PR Validation Integration - Wired *pip-audit* into *pr-validation.yml* as a matrix job calling the reusable workflow - Gated on `discover-python-projects` output with `fail-fast: false` and `changed-files-only: true` ### Dependency Review Hardening - Added license allowlist to *dependency-review.yml* covering MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC, 0BSD, BlueOak-1.0.0, CC0-1.0, Unlicense, CC-BY-4.0, CC-BY-3.0, PSF-2.0, and Python-2.0 - Enabled OSSF Scorecard display with warning threshold at level 3 ### Local Execution - Added *Invoke-PipAudit.ps1* with `Find-PythonProjects`, `Invoke-PipAuditForProject`, and `Start-PipAudit` functions - Used try/finally for temporary requirements file cleanup in `Invoke-PipAuditForProject` - Followed existing security-script conventions: comment-based help, copyright header, *CIHelpers.psm1* import, dot-source guard - Added `audit:pip` npm script to *package.json* ### Testing - Added 11 Pester tests in *Invoke-PipAudit.Tests.ps1* across three describe blocks - Covered project discovery (5 tests), audit execution (2 tests), and orchestration (4 tests) - Used TestDrive isolation, CI annotation assertions, and dot-source testing pattern ## Related Issue(s) Closes #1020 ## Type of Change Select all that apply: **Code & Documentation:** * [ ] Bug fix (non-breaking change fixing an issue) * [x] New feature (non-breaking change adding functionality) * [ ] Breaking change (fix or feature causing existing functionality to change) * [ ] Documentation update **Infrastructure & Configuration:** * [x] GitHub Actions workflow * [ ] Linting configuration (markdown, PowerShell, etc.) * [x] Security configuration * [ ] DevContainer configuration * [ ] Dependency update **AI Artifacts:** * [ ] Reviewed contribution with `prompt-builder` agent and addressed all feedback * [ ] Copilot instructions (`.github/instructions/*.instructions.md`) * [ ] Copilot prompt (`.github/prompts/*.prompt.md`) * [ ] Copilot agent (`.github/agents/*.agent.md`) * [ ] Copilot skill (`.github/skills/*/SKILL.md`) > Note for AI Artifact Contributors: > > * Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review `.github/agents/` before creating new ones. > * Skills: Must include both bash and PowerShell scripts. See [Skills](../docs/contributing/skills.md). > * Model Versions: Only contributions targeting the **latest Anthropic and OpenAI models** will be accepted. Older model versions (e.g., GPT-3.5, Claude 3) will be rejected. > * See [Agents Not Accepted](../docs/contributing/custom-agents.md#agents-not-accepted) and [Model Version Requirements](../docs/contributing/ai-artifacts-common.md#model-version-requirements). **Other:** * [x] Script/automation (`.ps1`, `.sh`, `.py`) * [ ] Other (please describe): ## Sample Prompts (for AI Artifact Contributions)  ## Testing All validation passed locally: - **Pester tests**: 11/11 passing across `Find-PythonProjects`, `Invoke-PipAuditForProject`, and `Start-PipAudit` describe blocks - **PSScriptAnalyzer**: Clean analysis on *Invoke-PipAudit.ps1* - **YAML validation**: All workflow files validated - **Copyright headers**: All new files include required headers ## Checklist ### Required Checks * [ ] Documentation is updated (if applicable) * [x] Files follow existing naming conventions * [x] Changes are backwards compatible (if applicable) * [x] Tests added for new functionality (if applicable) ### AI Artifact Contributions  * [ ] Used `/prompt-analyze` to review contribution * [ ] Addressed all feedback from `prompt-builder` review * [ ] Verified contribution follows common standards and type-specific requirements ### Required Automated Checks The following validation commands must pass before merging: * [x] Markdown linting: `npm run lint:md` * [x] Spell checking: `npm run spell-check` * [x] Frontmatter validation: `npm run lint:frontmatter` * [x] Skill structure validation: `npm run validate:skills` * [x] Link validation: `npm run lint:md-links` * [x] PowerShell analysis: `npm run lint:ps` --------- Co-authored-by: Bill Berry <wbery@microsoft.com>
diff --git a/.github/workflows/dependency-review.yml b/.github/workflows/dependency-review.yml
@@ -27,3 +27,10 @@ jobs:
         with:
           fail-on-severity: moderate
           comment-summary-in-pr: always
+          license-check: true
+          allow-licenses: >-
+            MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC,
+            0BSD, BlueOak-1.0.0, CC0-1.0, Unlicense,
+            CC-BY-4.0, CC-BY-3.0, PSF-2.0, Python-2.0
+          show-openssf-scorecard: true
+          warn-on-openssf-scorecard-level: 3
diff --git a/.github/workflows/pip-audit.yml b/.github/workflows/pip-audit.yml
@@ -0,0 +1,118 @@
+name: pip-audit
+
+on:
+  workflow_call:
+    inputs:
+      working-directory:
+        description: 'Directory containing the Python project to audit'
+        required: true
+        type: string
+      soft-fail:
+        description: 'When true, audit failures do not fail the build'
+        required: false
+        type: boolean
+        default: false
+      changed-files-only:
+        description: 'Only run audit if dependency files changed in this directory'
+        required: false
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+
+jobs:
+  pip-audit:
+    name: Audit Python Dependencies
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    defaults:
+      run:
+        shell: pwsh
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4.2.2
+        with:
+          persist-credentials: false
+          fetch-depth: 0
+
+      - name: Derive artifact name
+        id: meta
+        run: |
+          $name = '${{ inputs.working-directory }}' -replace '/', '-' -replace '^\.', '' -replace '^-', ''
+          "artifact-name=pip-audit-$name" >> $env:GITHUB_OUTPUT
+
+      - name: Check for dependency file changes
+        if: inputs.changed-files-only
+        run: |
+          $baseRef = if ($env:GITHUB_BASE_REF) { $env:GITHUB_BASE_REF } else { 'main' }
+          $workingDir = '${{ inputs.working-directory }}'
+          $changed = git diff --name-only --diff-filter=ACMR "origin/$baseRef...HEAD" -- $workingDir |
+            Where-Object { $_ -match '(pyproject\.toml|uv\.lock|requirements.*\.txt)$' }
+          if ($changed) {
+            "HAS_CHANGES=true" >> $env:GITHUB_ENV
+            $fileCount = @($changed).Count
+            Write-Output "Detected $fileCount changed dependency file(s)"
+          } else {
+            Write-Output "No dependency file changes detected under $workingDir"
+          }
+
+      - name: Install uv
+        if: "!inputs.changed-files-only || env.HAS_CHANGES == 'true'"
+        uses: astral-sh/setup-uv@e06108dd0aef18192324c70427afc47652e63a82 # v7.5.0
+        with:
+          version: "0.10.9"
+
+      - name: Setup Python
+        if: "!inputs.changed-files-only || env.HAS_CHANGES == 'true'"
+        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        with:
+          python-version: "3.12"
+
+      - name: Export locked dependencies
+        if: "!inputs.changed-files-only || env.HAS_CHANGES == 'true'"
+        run: |
+          uv export --format requirements-txt --no-hashes > "$env:RUNNER_TEMP/requirements.txt"
+          $lineCount = (Get-Content "$env:RUNNER_TEMP/requirements.txt" | Measure-Object).Count
+          Write-Output "Exported $lineCount locked dependencies"
+        working-directory: ${{ inputs.working-directory }}
+
+      - name: Run pip-audit
+        id: audit
+        if: "!inputs.changed-files-only || env.HAS_CHANGES == 'true'"
+        run: |
+          $name = '${{ steps.meta.outputs.artifact-name }}'
+          New-Item -ItemType Directory -Path "$env:GITHUB_WORKSPACE/logs" -Force | Out-Null
+          uvx pip-audit@2.10.0 `
+            -r "$env:RUNNER_TEMP/requirements.txt" `
+            --no-deps `
+            --format json `
+            -o "$env:GITHUB_WORKSPACE/logs/$name.json" `
+            --desc on `
+            --aliases on `
+            --progress-spinner off `
+            --strict
+          if ($LASTEXITCODE -ne 0) { "PIP_AUDIT_FAILED=true" >> $env:GITHUB_ENV }
+          $global:LASTEXITCODE = 0
+        working-directory: ${{ inputs.working-directory }}
+        continue-on-error: ${{ inputs.soft-fail }}
+
+      - name: Upload audit results
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v4.4.3
+        if: always() && (!inputs.changed-files-only || env.HAS_CHANGES == 'true')
+        with:
+          name: ${{ steps.meta.outputs.artifact-name }}
+          path: logs/${{ steps.meta.outputs.artifact-name }}.json
+          retention-days: 30
+          if-no-files-found: ignore
+
+      - name: Check results
+        if: "!inputs.soft-fail && (!inputs.changed-files-only || env.HAS_CHANGES == 'true')"
+        run: |
+          if ($env:PIP_AUDIT_FAILED -eq 'true') {
+            Write-Output '::error::pip-audit found vulnerabilities in Python dependencies'
+            exit 1
+          }
+          Write-Output 'pip-audit: no vulnerabilities found'
diff --git a/.github/workflows/pr-validation.yml b/.github/workflows/pr-validation.yml
@@ -145,6 +145,22 @@ jobs:
       matrix:
         directory: ${{ fromJson(needs.discover-python-projects.outputs.directories) }}
 
+  pip-audit:
+    name: "pip-audit (${{ matrix.directory }})"
+    needs: discover-python-projects
+    if: needs.discover-python-projects.outputs.has-projects == 'true'
+    strategy:
+      fail-fast: false
+      matrix:
+        directory: ${{ fromJson(needs.discover-python-projects.outputs.directories) }}
+    uses: ./.github/workflows/pip-audit.yml
+    permissions:
+      contents: read
+    with:
+      working-directory: ${{ matrix.directory }}
+      soft-fail: false
+      changed-files-only: true
+
   docusaurus-tests:
     name: Docusaurus Tests
     uses: ./.github/workflows/docusaurus-tests.yml
diff --git a/package.json b/package.json
@@ -19,6 +19,7 @@
     "lint:version-consistency": "pwsh -NoProfile -Command \"./scripts/security/Test-ActionVersionConsistency.ps1 -FailOnMismatch -Format Json -OutputPath logs/action-version-consistency-results.json\"",
     "lint:permissions": "pwsh -NoProfile -Command \"& './scripts/security/Test-WorkflowPermissions.ps1' -FailOnViolation\"",
     "lint:dependency-pinning": "pwsh -NoProfile -Command \"& './scripts/security/Test-DependencyPinning.ps1' -FailOnUnpinned\"",
+    "audit:pip": "pwsh -NoProfile -File scripts/security/Invoke-PipAudit.ps1",
     "lint:all": "npm run format:tables && npm run lint:md && npm run lint:ps && npm run lint:yaml && npm run lint:links && npm run lint:frontmatter && npm run lint:collections-metadata && npm run lint:marketplace && npm run lint:version-consistency && npm run lint:permissions && npm run lint:dependency-pinning && npm run lint:py && npm run validate:skills",
     "format:tables": "markdown-table-formatter \"**/*.md\"",
     "extension:prepare": "pwsh ./scripts/extension/Prepare-Extension.ps1",
diff --git a/scripts/security/Invoke-PipAudit.ps1 b/scripts/security/Invoke-PipAudit.ps1
@@ -0,0 +1,196 @@
+#!/usr/bin/env pwsh
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: MIT
+#Requires -Version 7.0
+
+<#
+.SYNOPSIS
+    Runs pip-audit against Python project dependencies for vulnerability scanning.
+
+.DESCRIPTION
+    Discovers Python projects containing pyproject.toml files, exports locked dependencies
+    via uv export, and runs pip-audit to check for known vulnerabilities. Results are written
+    as JSON to the logs directory.
+
+.PARAMETER Path
+    Root path to scan for Python projects. Defaults to repository root.
+
+.PARAMETER OutputPath
+    Directory for JSON results. Defaults to 'logs' under repository root.
+
+.PARAMETER FailOnVulnerability
+    Exit with error code if vulnerabilities are found. Default is false.
+
+.PARAMETER ExcludePaths
+    Comma-separated list of path patterns to exclude from scanning.
+
+.EXAMPLE
+    ./Invoke-PipAudit.ps1
+    Scan all Python projects and report results.
+
+.EXAMPLE
+    ./Invoke-PipAudit.ps1 -FailOnVulnerability
+    Scan and fail if any vulnerabilities are found.
+
+.EXAMPLE
+    ./Invoke-PipAudit.ps1 -Path ".github/skills/experimental/powerpoint"
+    Scan a specific Python project directory.
+#>
+
+[CmdletBinding()]
+param(
+    [Parameter()]
+    [string]$Path = (Split-Path -Parent (Split-Path -Parent $PSScriptRoot)),
+
+    [Parameter()]
+    [string]$OutputPath = (Join-Path (Split-Path -Parent (Split-Path -Parent $PSScriptRoot)) 'logs'),
+
+    [Parameter()]
+    [switch]$FailOnVulnerability,
+
+    [Parameter()]
+    [string[]]$ExcludePaths = @()
+)
+
+$ErrorActionPreference = 'Stop'
+
+Import-Module (Join-Path $PSScriptRoot '../lib/Modules/CIHelpers.psm1') -Force
+
+function Find-PythonProjects {
+    <#
+    .SYNOPSIS
+        Discovers Python projects containing pyproject.toml files.
+    #>
+    [CmdletBinding()]
+    param(
+        [Parameter(Mandatory)]
+        [string]$SearchPath,
+
+        [Parameter()]
+        [string[]]$Exclude = @()
+    )
+
+    @(Get-ChildItem -Path $SearchPath -Recurse -Force -Filter pyproject.toml |
+        Where-Object { $_.FullName -notmatch 'node_modules' } |
+        ForEach-Object { $_.DirectoryName } |
+        Where-Object {
+            $dir = $_
+            $excluded = $false
+            foreach ($pattern in $Exclude) {
+                if ($dir -like "*$pattern*") { $excluded = $true; break }
+            }
+            -not $excluded
+        } |
+        Sort-Object)
+}
+
+function Invoke-PipAuditForProject {
+    <#
+    .SYNOPSIS
+        Runs pip-audit against a single Python project directory.
+    #>
+    [CmdletBinding()]
+    param(
+        [Parameter(Mandatory)]
+        [string]$ProjectPath,
+
+        [Parameter(Mandatory)]
+        [string]$OutputPath
+    )
+
+    $name = (Resolve-Path -Relative $ProjectPath) -replace '[\\/]', '-' -replace '^\.', '' -replace '^-', ''
+    $requirementsFile = Join-Path ([System.IO.Path]::GetTempPath()) "requirements-$name.txt"
+    $resultsFile = Join-Path $OutputPath "pip-audit-$name.json"
+
+    Write-Host "Auditing: $ProjectPath"
+
+    # Export locked dependencies
+    Push-Location $ProjectPath
+    try {
+        uv export --format requirements-txt --no-hashes > $requirementsFile
+    } finally {
+        Pop-Location
+    }
+
+    # Run pip-audit; finally block ensures temp file cleanup on terminating errors
+    try {
+        uvx pip-audit@2.10.0 `
+            -r $requirementsFile `
+            --no-deps `
+            --format json `
+            -o $resultsFile `
+            --desc on `
+            --aliases on `
+            --progress-spinner off `
+            --strict
+
+        $exitCode = $LASTEXITCODE
+    } finally {
+        if (Test-Path $requirementsFile) { Remove-Item $requirementsFile }
+    }
+
+    if ($exitCode -ne 0) {
+        Write-Host "::warning::Vulnerabilities found in $ProjectPath"
+        return $true  # has vulnerabilities
+    }
+
+    Write-Host "No vulnerabilities found in $ProjectPath"
+    return $false
+}
+
+function Start-PipAudit {
+    <#
+    .SYNOPSIS
+        Orchestrates pip-audit scanning across discovered Python projects.
+    .OUTPUTS
+        System.Int32 - 0 for success, 1 when vulnerabilities found and FailOnVulnerability is set.
+    #>
+    [CmdletBinding()]
+    param(
+        [Parameter(Mandatory)]
+        [string]$SearchPath,
+
+        [Parameter(Mandatory)]
+        [string]$OutputPath,
+
+        [Parameter()]
+        [switch]$FailOnVulnerability,
+
+        [Parameter()]
+        [string[]]$ExcludePaths = @()
+    )
+
+    $projects = @(Find-PythonProjects -SearchPath $SearchPath -Exclude $ExcludePaths)
+
+    if ($projects.Count -eq 0) {
+        Write-Host 'No Python projects found'
+        return 0
+    }
+
+    Write-Host "Found $($projects.Count) Python project(s)"
+
+    New-Item -ItemType Directory -Path $OutputPath -Force | Out-Null
+
+    $hasVulnerabilities = $false
+
+    foreach ($project in $projects) {
+        if (Invoke-PipAuditForProject -ProjectPath $project -OutputPath $OutputPath) {
+            $hasVulnerabilities = $true
+        }
+    }
+
+    Write-Host "Results written to $OutputPath"
+
+    if ($hasVulnerabilities -and $FailOnVulnerability) {
+        Write-Host '::error::pip-audit found vulnerabilities in one or more Python projects'
+        return 1
+    }
+
+    return 0
+}
+
+# Dot-source guard: skip main execution when dot-sourced for testing
+if ($MyInvocation.InvocationName -ne '.') {
+    $result = Start-PipAudit -SearchPath $Path -OutputPath $OutputPath -FailOnVulnerability:$FailOnVulnerability -ExcludePaths $ExcludePaths
+    if ($result -ne 0) { exit $result }
+}
diff --git a/scripts/tests/security/Invoke-PipAudit.Tests.ps1 b/scripts/tests/security/Invoke-PipAudit.Tests.ps1
