Skip to content

Conversation

@cgwalters
Copy link
Collaborator

@cgwalters cgwalters commented Nov 5, 2025

Closes: #1624

@bootc-bot bootc-bot bot requested a review from gursewak1997 November 5, 2025 17:42
@cgwalters
Copy link
Collaborator Author

cgwalters commented Nov 5, 2025

OK hum that's interesting, all tests passed except /tmt/plans/integration/test-27-custom-selinux-policy....because of a reboot timeout? Weird...

(Also man digging through tmt logs to find the actual failure is really just terrible...it looks like https://gitlab.com/testing-farm/oculus is the thing that fixes all this up for testing-farm?)

In debugging it seems like something is going wrong with tmt-reboot, if I ssh into the guest and just run reboot then the test passes.

@cgwalters cgwalters force-pushed the test-bcvk branch 3 times, most recently from 99e2a10 to c2d0dbf Compare November 6, 2025 14:12
@@ -0,0 +1,35 @@
# TMT soft-reboot limitation

TMT does not currently support systemd soft-reboots. It detects reboots by checking
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Man did I get burned by this. It's very subtle because basically tmt has a another bug where tmt-reboot with testcloud for some reason goes out to the hypervisor and "physically" shuts down and restarts...bypassing the in-guest reboot logic entirely.

But of course, we're not doing that with bcvk because 1) there's no magic backchannel to do so 2) why would we it doesn't make sense

@cgwalters
Copy link
Collaborator Author

  • test centos-9 = centos-10

Oh man am I so glad I had that last minute thought while refactoring to cross-check that we're actually building for the specified target OS...

@cgwalters cgwalters force-pushed the test-bcvk branch 2 times, most recently from 1c64ff6 to 281ecfb Compare November 6, 2025 17:20
Signed-off-by: Colin Walters <[email protected]>
TMT does not support systemd soft-reboots - it only detects reboots
by checking if /proc/stat btime changes, which doesn't happen during
soft-reboots. This caused test-custom-selinux-policy to hang when
running with bcvk (which allows actual soft-reboots), while it
accidentally passed with testcloud (which forced full VM reboots).

Add bug-soft-reboot.md documenting this limitation and update both
test files to reference it. Also remove --soft-reboot=auto from
test-custom-selinux-policy since we can't test it with TMT anyway.

Assisted-by: Claude Code (Sonnet 4.5)
Signed-off-by: Colin Walters <[email protected]>
Part 1: Use bcvk

For local tests, right now testcloud+tmt doesn't support UEFI, see
teemtee/tmt#4203

This is a blocker for us doing more testing with UKIs.

In this patch we switch to provisioning VMs with bcvk, which
fixes this - but beyond that a really compelling thing about
this is that bcvk is *also* designed to be ergonomic and efficient
beyond just being a test runner, with things like virtiofs
mounting of host container storage, etc.

In other words, bcvk is the preferred way to run local virt
with bootc, and this makes our TMT tests use it.

Now a major downside of this though is we're effectively
implementing a new "provisioner" for tmt (bypassing the
existing `virtual`). In the more medium term I think we
want to add `bcvk` as a provisioner option to tmt.

Anyways for now, this works by discovers test plans via `tmt plan ls`,
spawning a separate VM per test, and then using uses tmt's connect
provisioner to run tests targeting these externally provisioned
systems.

Part 2: Rework the Justfile and Dockerfile

This adds `base` and `variant` arguments which are propagated through
the system, and we have a new `variant` for sealed composefs.

The readonly tests now pass with composefs.

Drop the continuous repo tests...as while we could keep
that it's actually a whole *other* entry in this matrix.

Assisted-by: Claude Code (Sonnet 4.5)
Signed-off-by: Colin Walters <[email protected]>
I'm changing the default fs for Fedora in our CI to be xfs arbitrarily.
This code SHOULD work on non-fsverity hosts, and the other
code path in `tests/build-sealed` does.

Also, the remainder of the stuff was dead code so just drop it.

Signed-off-by: Colin Walters <[email protected]>
@cgwalters
Copy link
Collaborator Author

Oh man so close, the last bug seems to be that the two jobs for centos-10 (one ostree, one composefs) were trying to save the same artifact ID, that should be fixed...

@cgwalters cgwalters marked this pull request as ready for review November 6, 2025 22:09
@cgwalters cgwalters enabled auto-merge (rebase) November 6, 2025 22:58
@cgwalters
Copy link
Collaborator Author

🎉 it's passing!
@gursewak1997 can you review?

# To run an individual test, pass it as an argument like:
# `just test-tmt readonly`
test-tmt *ARGS: build-integration-test-image
cargo xtask run-tmt --env=BOOTC_variant={{variant}} localhost/bootc-integration {{ARGS}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kinda confused me, as in we are using cargo xtask run-tmt (Rust implementation using bcvk)

Copy link
Contributor

@gursewak1997 gursewak1997 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good from the initial look

@cgwalters cgwalters merged commit 3181bb5 into bootc-dev:main Nov 7, 2025
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tmt/testing followups phase 2: flexible provisioning and build

2 participants