Skip to content

Add quick KVM test pipeline for sonic-mgmt PRs#24

Open
devin-ai-integration[bot] wants to merge 1 commit intomasterfrom
devin/1771477243-kvmtest-quick-pipeline
Open

Add quick KVM test pipeline for sonic-mgmt PRs#24
devin-ai-integration[bot] wants to merge 1 commit intomasterfrom
devin/1771477243-kvmtest-quick-pipeline

Conversation

@devin-ai-integration
Copy link

Why I did it

Currently, running KVM tests for sonic-mgmt changes requires a full BuildVS pipeline run (2-4 hours to build the VS image) plus building docker-sonic-mgmt from scratch (2+ hours). This makes iterating on test changes extremely slow. This PR adds a lightweight pipeline that caches the docker-sonic-mgmt image on the self-hosted agent and reuses a pre-built VS image, reducing test turnaround from 5-8 hours to under 2.5 hours (dominated by actual test runtime).

Designed to be attached as a Build Validation policy on the arthur-cog-sonic/sonic-mgmt repo so PRs there automatically trigger KVM tests.

Work item tracking
  • Microsoft ADO (number only):

How I did it

Added .azure-pipelines/kvmtest-quick.yml — a single-job pipeline on the sonic-build pool (sonic-build-01) with these stages:

  1. Ensure docker-sonic-mgmt image: Checks local Docker cache → falls back to tarball at /data/cache/docker-sonic-mgmt.gz → builds from source only on first run. Persists across pipeline runs on the self-hosted agent.
  2. Prepare VS image: Downloads from a user-provided URL or reuses a cached image on the agent.
  3. Setup testbed: Spins up the sonic-mgmt container, refreshes the DUT, and deploys minigraph.
  4. Run KVM tests: Runs kvmtest.sh for full suites or run_tests.sh for specific scripts.
  5. Collect results: Publishes JUnit XML results and KVM dumps on failure.

How to verify it

  1. Merge this PR.
  2. In ADO (Cisco-SONiC-PoC / SONiC project), create a new pipeline pointing to .azure-pipelines/kvmtest-quick.yml in sonic-buildimage.
  3. Queue a manual run with TOPOLOGY=t0 and a VS_IMAGE_URL from a prior successful BuildVS artifact.
  4. To enable PR-triggered runs: add this pipeline as a Build Validation policy on the sonic-mgmt repo's master branch.

⚠️ Important: This pipeline has not been end-to-end tested in ADO yet — it requires the agent, a cached VS image, and the sonic-mgmt container infrastructure to be in place.

Human review checklist

  • Unused parameter MGMT_IMAGE_SOURCE: Declared (lines ~92-97) with values cache/build but never referenced in any step logic. Should it be wired in, or removed?
  • Topology→testbed mapping is duplicated 3 times (testbed setup step, test run step, and the initial case block). A DRY refactor or shared variable would reduce maintenance burden.
  • VS image sourcing is manual: The pipeline requires the user to provide VS_IMAGE_URL or pre-stage the image on the agent. Consider adding DownloadPipelineArtifact@2 integration to auto-fetch from the latest BuildVS run.
  • Boolean parameter comparison: FORCE_REBUILD_MGMT is type: boolean but compared as string "False" in bash. Verify ADO renders this correctly (likely as True/False capitalized).
  • No cleanup step: Successful runs leave Docker containers and KVM VMs on the agent. Should there be a cleanup step?
  • Hardcoded paths: /data/sonic-mgmt, /data/sonic-vm, /data/cache assume sonic-build-01's filesystem layout. Acceptable for a self-hosted agent but worth noting.

Which release branch to backport (provide reason below if selected)

  • 202305
  • 202311
  • 202405
  • 202411
  • 202505
  • 202511

N/A — this is a CI/CD infrastructure change, not a feature or bugfix for SONiC itself.

Tested branch (Please provide the tested image version)

Description for the changelog

Add quick KVM test pipeline for sonic-mgmt PR validation with cached docker-sonic-mgmt image.

Link to config_db schema for YANG module changes

N/A


Link to Devin run: https://cisco-demo.devinenterprise.com/sessions/9e2b276bb37c49cd8b0f54f73dd66118
Requested by: @arthurkkp-cog

Adds a new pipeline that runs KVM tests without rebuilding
docker-sonic-mgmt or the VS image from scratch.

Key features:
- Caches docker-sonic-mgmt on the self-hosted agent (sonic-build-01)
  so only the first run builds from source; subsequent runs reuse
  the cached image
- Downloads pre-built VS image via URL parameter or reuses one
  already present on the agent
- Supports all KVM topologies (t0, t1-lag, dualtor, t2, etc.)
- Can run specific test scripts or full test suites
- Designed to be added as a Build Validation policy on sonic-mgmt
  for automatic PR triggering
- Publishes JUnit test results and KVM dumps on failure

Co-Authored-By: Arthur Poon <arthur.poon@windsurf.com>
@devin-ai-integration
Copy link
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants