Skip to content

Commit 2419732

Browse files
lsm5lsm5
committed
[skip-ci] TMT: system tests
This commit introduces TMT test jobs triggered via packit to run system tests on testing-farm infrastructure. Tests are run for Fedora 41, 42 and rawhide on x86_64. The same test plan will be reused by Fedora for bodhi, zuul and fedora-ci gating tests. Packit will handle syncing of test plan and sources from upstream to downstream. Packit failure notification has also been updated to be less noisy and let people know they are free to ignore any failures. TODO: 1. Enable jobs for CentOS Stream and aarch64 envs. 2. Enable separate set of jobs for release branches as they need to be tested with official distro packages, not with bleeding-edge packages. Signed-off-by: Lokesh Mandvekar <[email protected]>
1 parent 804ad4e commit 2419732

File tree

9 files changed

+273
-2
lines changed

9 files changed

+273
-2
lines changed

‎.packit.yaml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,27 @@
55
downstream_package_name: podman
66
upstream_tag_template: v{version}
77

8+
# These files get synced from upstream to downstream (Fedora / CentOS Stream) on every
9+
# propose-downstream job. This is done so tests maintained upstream can be run
10+
# downstream in Zuul CI and Bodhi.
11+
# Ref: https://packit.dev/docs/configuration#files_to_sync
12+
files_to_sync:
13+
- src: rpm/gating.yaml
14+
dest: gating.yaml
15+
delete: true
16+
- src: plans/
17+
dest: plans/
18+
delete: true
19+
mkpath: true
20+
- src: test/tmt/
21+
dest: test/tmt/
22+
delete: true
23+
mkpath: true
24+
- src: .fmf/
25+
dest: .fmf/
26+
delete: true
27+
- .packit.yaml
28+
829
packages:
930
podman-fedora:
1031
pkg_tool: fedpkg
@@ -80,6 +101,22 @@ jobs:
80101
project: podman-next
81102
enable_net: true
82103

104+
# Tests on Fedora
105+
- job: tests
106+
trigger: pull_request
107+
packages: [podman-fedora]
108+
notifications: *packit_generic_failure_notification
109+
targets:
110+
- fedora-rawhide
111+
- fedora-42
112+
- fedora-41
113+
tmt_plan: "/plans/system/*"
114+
tf_extra_params:
115+
environments:
116+
- artifacts:
117+
- type: repository-file
118+
id: https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/repo/fedora-$releasever/rhcontainerbot-podman-next-fedora-$releasever.repo
119+
83120
- job: tests
84121
identifier: cockpit-revdeps
85122
trigger: pull_request

‎plans/system.fmf‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
discover:
2+
how: fmf
3+
4+
execute:
5+
how: tmt
6+
7+
prepare:
8+
- how: shell
9+
script: modprobe null_blk nr_devices=1
10+
order: 5
11+
- when: distro == centos-stream or distro == rhel
12+
how: shell
13+
script: |
14+
dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(rpm --eval '%{?rhel}').noarch.rpm
15+
dnf -y config-manager --set-enabled epel
16+
order: 10
17+
- when: initiator == packit
18+
how: shell
19+
script: |
20+
COPR_REPO_FILE="/etc/yum.repos.d/*podman-next*.repo"
21+
if compgen -G $COPR_REPO_FILE > /dev/null; then
22+
sed -i -n '/^priority=/!p;$apriority=1' $COPR_REPO_FILE
23+
fi
24+
dnf -y upgrade --allowerasing
25+
order: 20
26+
27+
adjust+:
28+
- enabled: false
29+
when: revdeps == yes
30+
31+
provision:
32+
how: artemis
33+
hardware:
34+
memory: ">= 16 GB"
35+
cpu:
36+
cores: ">= 4"
37+
threads: ">=8"
38+
disk:
39+
- size: ">= 512 GB"
40+
41+
/local-root:
42+
summary: Local rootful tests
43+
discover+:
44+
filter: 'tag:local & tag:root'
45+
46+
/local-rootless:
47+
summary: Local rootless tests
48+
discover+:
49+
filter: 'tag:local & tag:rootless'
50+
51+
/remote-root:
52+
summary: Remote rootful tests
53+
discover+:
54+
filter: 'tag:remote & tag:root'
55+
56+
/remote-rootless:
57+
summary: Remote rootless tests
58+
discover+:
59+
filter: 'tag:remote & tag:rootless'

‎plans/tmt.fmf‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
summary: Run tmt container provision test (downstream only)
2+
3+
enabled: false
4+
adjust+:
5+
- enabled: true
6+
when: initiator != packit and distro != rhel
7+
8+
discover:
9+
how: fmf
10+
filter: 'tag:tmt & tag:downstream'
11+
12+
execute:
13+
how: tmt
14+
15+
prepare:
16+
- when: distro == centos-stream or distro == rhel
17+
how: shell
18+
script: |
19+
dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(rpm --eval '%{?rhel}').noarch.rpm
20+
dnf -y config-manager --set-enabled epel
21+
order: 10

‎plans/toolbox.fmf‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
summary: Run toolbox tests (downstream only)
2+
3+
enabled: false
4+
adjust+:
5+
- enabled: true
6+
when: initiator != packit and distro == fedora
7+
8+
provision:
9+
how: artemis
10+
hardware:
11+
memory: ">= 16 GB"
12+
cpu:
13+
cores: ">= 4"
14+
threads: ">=8"
15+
disk:
16+
- size: ">= 512 GB"
17+
18+
prepare:
19+
- name: packages
20+
how: install
21+
package: [toolbox-tests]
22+
23+
discover:
24+
how: fmf
25+
url: https://src.fedoraproject.org/rpms/toolbox
26+
ref: "rawhide"
27+
28+
execute:
29+
how: tmt

‎rpm/gating.yaml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
--- !Policy
2+
product_versions:
3+
- fedora-*
4+
decision_contexts:
5+
- bodhi_update_push_stable
6+
- bodhi_update_push_testing
7+
subject_type: koji_build
8+
rules:
9+
- !PassingTestCaseRule {test_case_name: fedora-ci.koji-build.tier0.functional}
10+
11+
# recipients: jnovy, lsm5, santiago
12+
--- !Policy
13+
product_versions:
14+
- rhel-*
15+
decision_context: osci_compose_gate
16+
rules:
17+
- !PassingTestCaseRule {test_case_name: osci.brew-build.tier0.functional}

‎rpm/podman.spec‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@
1313
%define build_with_btrfs 1
1414
# qemu-system* isn't packageed for CentOS Stream / RHEL
1515
%define qemu 1
16+
# bats is included in the default repos (No epel/copr etc.)
17+
%define distro_bats 1
1618
%endif
1719

1820
%if %{defined copr_username}
@@ -139,7 +141,7 @@ pages and %{name}.
139141
Summary: Tests for %{name}
140142

141143
Requires: %{name} = %{epoch}:%{version}-%{release}
142-
%if %{defined fedora}
144+
%if %{defined distro_bats}
143145
Requires: bats
144146
%endif
145147
Requires: attr
@@ -157,7 +159,8 @@ Requires: xfsprogs
157159
%description tests
158160
%{summary}
159161

160-
This package contains system tests for %{name}
162+
This package contains system tests for %{name}. Only intended to be used for
163+
gating tests. Not supported for end users / customers.
161164

162165
%package remote
163166
Summary: (Experimental) Remote client for managing %{name} containers

‎test/tmt/system.fmf‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
require:
2+
- podman-tests
3+
- psmisc
4+
5+
environment:
6+
# PODMAN_TESTING envvar is set in system.sh
7+
PODMAN: /usr/bin/podman
8+
QUADLET: /usr/libexec/podman/quadlet
9+
ROOTLESS_USER: "fedora"
10+
adjust+:
11+
- when: distro == centos-stream
12+
environment+:
13+
ROOTLESS_USER: "ec2-user"
14+
- when: distro == rhel
15+
environment+:
16+
ROOTLESS_USER: "cloud-user"
17+
18+
/local-root:
19+
tag: [ local, root ]
20+
summary: local rootful test
21+
test: bash ./system.sh
22+
duration: 30m
23+
24+
/local-rootless:
25+
tag: [ local, rootless ]
26+
summary: rootless test
27+
test: bash ./system.sh rootless
28+
duration: 30m
29+
30+
/remote-root:
31+
tag: [ remote, root ]
32+
summary: remote rootful test
33+
test: bash ./system.sh
34+
duration: 30m
35+
environment+:
36+
PODMAN: /usr/bin/podman-remote
37+
require+:
38+
- podman-remote
39+
40+
/remote-rootless:
41+
tag: [ remote, rootless ]
42+
summary: remote rootless test
43+
test: bash ./system.sh rootless
44+
duration: 30m
45+
environment+:
46+
PODMAN: /usr/bin/podman-remote
47+
require+:
48+
- podman-remote

‎test/tmt/system.sh‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
#!/usr/bin/env bash
2+
3+
set -exo pipefail
4+
5+
uname -r
6+
7+
loginctl enable-linger "$ROOTLESS_USER"
8+
9+
rpm -q \
10+
aardvark-dns \
11+
buildah \
12+
conmon \
13+
container-selinux \
14+
containers-common \
15+
criu \
16+
crun \
17+
netavark \
18+
passt \
19+
podman \
20+
podman-tests \
21+
skopeo \
22+
slirp4netns \
23+
systemd
24+
25+
export system_service_cmd="/usr/bin/podman system service --timeout=0 &"
26+
export test_cmd="whoami && cd /usr/share/podman/test/system && PODMAN_TESTING=/usr/bin/podman-testing bats ."
27+
28+
if [[ -z $1 ]]; then
29+
if [[ $PODMAN == "/usr/bin/podman-remote" ]]; then
30+
eval "$system_service_cmd"
31+
fi
32+
eval "$test_cmd"
33+
elif [[ $1 == "rootless" ]]; then
34+
if [[ $PODMAN == "/usr/bin/podman-remote" ]]; then
35+
su - "$ROOTLESS_USER" -c "eval $system_service_cmd"
36+
fi
37+
su - "$ROOTLESS_USER" -c "eval $test_cmd"
38+
fi
39+
40+
# Kill all podman processes for remote tests
41+
if [[ $PODMAN == "/usr/bin/podman-remote" ]]; then
42+
killall -q podman
43+
fi
44+
exit 0

‎test/tmt/tmt.fmf‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
enabled: false
2+
adjust:
3+
enabled: true
4+
when: initiator != packit && distro != rhel
5+
summary: Make sure that TMT container provision works
6+
tag: [downstream]
7+
require:
8+
- tmt+provision-container
9+
test:
10+
tmt run --verbose --remove
11+
provision --how container --image fedora
12+
login --command 'cat /etc/os-release'
13+
finish

0 commit comments

Comments
 (0)