Skip to content

Comments

tests: Update tests_zone to do bootc end-to-end validation#280

Merged
martinpitt merged 5 commits intolinux-system-roles:mainfrom
martinpitt:qemu-bootc
Jun 3, 2025
Merged

tests: Update tests_zone to do bootc end-to-end validation#280
martinpitt merged 5 commits intolinux-system-roles:mainfrom
martinpitt:qemu-bootc

Conversation

@martinpitt
Copy link
Contributor

@martinpitt martinpitt commented Jun 3, 2025

That test covers a lot of settings but only has one scenario, so it's
fitting for end-to-end testing.

See https://issues.redhat.com/browse/RHEL-78157


The other commits are identical to linux-system-roles/sudo#59 and linux-system-roles/.github#114

Summary by Sourcery

Implement end-to-end bootc validation for tests_zone, update QEMU integration and other CI workflows to support bootc scenarios, and bump tox-lsr to v3.10.0 across all pipelines

New Features:

  • Introduce end-to-end validation of bootc images in the QEMU integration test workflow

Enhancements:

  • Update tests_zone.yml to conditionally apply bootc validation flags and add a QEMU deployment step via bootc-buildah-qcow.sh

CI:

  • Bump tox-lsr dependency to v3.10.0 in ansible-lint, ansible-managed-var-comment, ansible-test, and python-unit-test workflows
  • Add podman 5.x pinning and installation steps for bootc-compatible testing in the qemu-kvm-integration-tests workflow

Tests:

  • Exclude bootc-specific tests from default QEMU runs and route them through the new bootc validation step

@sourcery-ai
Copy link

sourcery-ai bot commented Jun 3, 2025

Reviewer's Guide

This PR enhances CI workflows and test playbooks to perform bootc end-to-end validation: it bumps the tox-lsr dependency, injects a Podman-5 compatibility hack, adds a new QEMU-based validation step, and augments the existing tests_zone playbook with conditional logic and deployment support for bootc scenarios.

Sequence Diagram: tests_zone.yml Execution in bootc Scenario

sequenceDiagram
    participant AC as Ansible Controller
    participant TG as Target (bootc Environment)

    AC->>+TG: Initiate tests_zone.yml playbook
    TG-->>AC: Acknowledge
    AC->>AC: Check if target is bootc environment
    Note over AC: Conditional logic for bootc
    AC->>+TG: Perform bootc-specific deployment steps
    TG-->>AC: Deployment successful
    AC->>+TG: Execute zone configuration tests
    TG-->>AC: Test results
    AC->>AC: Process validation results
Loading

Flow Diagram: Conditional Logic in tests_zone.yml for bootc

graph TD
    Start((Start tests_zone.yml)) --> IsBootc{Is bootc environment?};
    IsBootc -- Yes --> BootcDeployment[Perform bootc-specific<br/>deployment tasks];
    IsBootc -- No --> StandardDeployment[Perform standard<br/>deployment tasks];
    BootcDeployment --> CommonTests[Execute common zone tests];
    StandardDeployment --> CommonTests;
    CommonTests --> End((End playbook));
Loading

File-Level Changes

Change Details Files
Bump tox-lsr dependency version across GitHub workflows
  • Updated pip install commands to reference tox-lsr@3.10.0
.github/workflows/qemu-kvm-integration-tests.yml
.github/workflows/ansible-lint.yml
.github/workflows/ansible-managed-var-comment.yml
.github/workflows/ansible-test.yml
.github/workflows/python-unit-test.yml
Add Podman-5 compatibility hack for bootc scenarios
  • Inject Plucky apt sources and APT pinning for Podman and container packages
  • Conditionally install Podman 5.x runtime dependencies when running '-bootc' images
.github/workflows/qemu-kvm-integration-tests.yml
Introduce bootc end-to-end validation step in CI
  • Add a 'Run bootc validation tests in QEMU' step that loops over built QCOW2 images
  • Use __bootc_validation flag to drive per-test pass/fail logging and outcome
.github/workflows/qemu-kvm-integration-tests.yml
Enhance tests_zone playbook for bootc validation
  • Wrap existing fail and cleanup tasks with when: not __bootc_validation conditions
  • Introduce a 'Create QEMU deployment during bootc end-to-end test' task under buildah connection
tests/tests_zone.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@codecov
Copy link

codecov bot commented Jun 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 58.41%. Comparing base (2d7c4ba) to head (c068aa4).
Report is 87 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #280      +/-   ##
==========================================
- Coverage   61.09%   58.41%   -2.69%     
==========================================
  Files           2        2              
  Lines         910     1188     +278     
==========================================
+ Hits          556      694     +138     
- Misses        354      494     +140     
Flag Coverage Δ
sanity ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @martinpitt - I've reviewed your changes - here's some feedback:

  • Consider extracting the repeated tox-lsr version bump into a shared variable or install script so you don’t have to update every workflow file whenever the version changes.
  • The sed-based hack to pin and upgrade Podman to 5.x is fragile—see if you can use an official backport PPA or the upstream container tools package stream instead of rewriting sources.list in your CI.
  • In the new QEMU bootc validation loop, guard the ls tests/tmp/*/qcow2/disk.qcow2 call (e.g. with || continue or an existence check) to avoid failing the step when no images are found.
Here's what I looked at during the review
  • 🟢 General issues: all looks good
  • 🟢 Security: all looks good
  • 🟢 Review instructions: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@martinpitt
Copy link
Contributor Author

meh codespell, same issue as in sudo. I'll cherry-pick the same fix.

@martinpitt
Copy link
Contributor Author

F41 failure is due to osbuild/bootc-image-builder#949 , will be fixed tomorrow-ish when a new bootc base image picks up https://bodhi.fedoraproject.org/updates/FEDORA-2025-f68c3fced6

@martinpitt martinpitt requested review from richm and spetrosi June 3, 2025 09:20
Ubuntu 24.04's podman 4.9.3 does not work with the bootc-image-builder
container: Bind-mounting the container storage into it makes the
container's podman 5 fail with a storage error.

Thus install podman 5.4 from Ubuntu 25.04 to regain compatibility. This
is rather hackish, but as that is a stable release, it should not break
in the future (wrt. library dependencies and such).

Ignore "passt" codespell complaint. This *is* the correct name for our
test dependency.
These tests run the role during a bootc container image build, deploy
the container into a QEMU VM, boot that, and validate the expected
configuration there. They run in two different tox environments, and
thus have to be run in two steps (preparation in buildah, validation in
QEMU). The preparation is expected to output a qcow2 image in
`tests/tmp/TESTNAME/qcow2/disk.qcow2`, i.e. the output structure of
<https://github.com/osbuild/bootc-image-builder>.

There are two possibilities:

 * Have separate bootc end-to-end tests. These are tagged with
   `tests::bootc-e2` and are skipped in the normal qemu-* scenarios.
   They run as part of the container-* ones.

 * Modify an existing test: These need to build a qcow2 image exactly
   *once* (via calling `bootc-buildah-qcow.sh`) and skip setup/cleanup
   and role invocations in validation mode, i.e. when
   `__bootc_validation` is true.

In the container scenario, run the QEMU validation as a separate step in
the workflow.

See https://issues.redhat.com/browse/RHEL-88396
That test covers a lot of settings but only has one scenario, so it's
fitting for end-to-end testing.

See https://issues.redhat.com/browse/RHEL-78157
@martinpitt martinpitt merged commit f5d8e6f into linux-system-roles:main Jun 3, 2025
27 of 28 checks passed
@martinpitt martinpitt deleted the qemu-bootc branch June 3, 2025 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants