Skip to content

Commit 0430d8d

Browse files
committed
ci: Add test plan that runs CI tests and customize it for each role
* Calculate number of managed nodes with this formula: (( number_of_test_playbooks / 10 + 1 )) * Add README explaining how to run the plan locally and remotely Signed-off-by: Sergei Petrosian <[email protected]>
1 parent 62d3bdc commit 0430d8d

File tree

3 files changed

+77
-15
lines changed

3 files changed

+77
-15
lines changed

.codespellrc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,6 @@ check-hidden = true
55
ignore-multiline-regex = codespell:ignore-begin.*codespell:ignore-end
66
ignore-words = .codespell_ignores
77
# skip-file is not available https://github.com/codespell-project/codespell/pull/2759
8+
# .pandoc_template.html5 contains a typo in Licence that we shouldn't edit
9+
# .README.html is generated from README.md automatically - no need to check spelling
810
skip = .pandoc_template.html5,.README.html,./tests/files/client.key.nocrypt,./tests/files/client.pem

plans/README-plans.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Introduction CI Testing Plans
2+
3+
Linux System Roles CI runs [tmt](https://tmt.readthedocs.io/en/stable/index.html) test plans in [Testing farm](https://docs.testing-farm.io/Testing%20Farm/0.1/index.html) with the [tft.yml](https://github.com/linux-system-roles/network/blob/main/.github/workflows/tft.yml) GitHub workflow.
4+
5+
The `plans/test_playbooks_parallel.fmf` plan is a test plan that runs test playbooks in parallel on multiple managed nodes.
6+
`plans/test_playbooks_parallel.fmf` is generated centrally from `https://github.com/linux-system-roles/.github/`.
7+
The automation calculates the number of managed nodes to provision with this formula:
8+
9+
```plain
10+
number-of-test-playbooks / 10 + 1
11+
```
12+
13+
The `plans/test_playbooks_parallel.fmf` plan does the following steps:
14+
15+
1. Provisions systems to be used as a control node and as managed nodes.
16+
2. Does the required preparation on systems.
17+
3. For the given role and the given PR, runs the general test from [test.sh](https://github.com/linux-system-roles/tft-tests/blob/main/tests/general/test.sh).
18+
19+
The [tft.yml](https://github.com/linux-system-roles/network/blob/main/.github/workflows/tft.yml) workflow runs the above plan and uploads the results to our Fedora storage for public access.
20+
This workflow uses Testing Farm's Github Action [Schedule tests on Testing Farm](https://github.com/marketplace/actions/schedule-tests-on-testing-farm).
21+
22+
## Running Tests
23+
24+
You can run tests locally with the `tmt try` cli or remotely in Testing Far.
25+
26+
### Running Tests Locally
27+
28+
1. Install `tmt` as described in [Installation](https://tmt.readthedocs.io/en/stable/stories/install.html).
29+
2. Change to the role repository directory.
30+
3. Modify `plans/test_playbooks_parallel.fmf` to suit your requirements:
31+
1. Due to [issue #3138](https://github.com/teemtee/tmt/issues/3138), comment out all managed nodes except for one.
32+
2. Optionally modify environment variables to, e.g. run only specified test playbooks by modifying `SYSTEM_ROLES_ONLY_TESTS`.
33+
4. Enter `tmt try -p plans/test_playbooks_parallel <platform>`.
34+
This command identifies the `plans/test_playbooks_parallel.fmf` plan and provisions local VMs, a control node and a managed node.
35+
5. tmt try is in development and does not identify tests from URL automatically, so after provisioning the machines, you must type `t`, `p`, `t` from the interactive prompt to identify tests, run preparation steps, and run the tests.
36+
37+
### Running in Testing Farm
38+
39+
1. Install `testing-farm` as described in [Installation](https://gitlab.com/testing-farm/cli/-/blob/main/README.adoc#user-content-installation).
40+
2. Change to the role repository directory.
41+
3. If you want to run tests with edits in your branch, you need to commit and push changes first to some branch.
42+
4. You can uncomment "Inject your ssh public key to test systems" discover step in the plan if you want to troubleshoot tests by SSHing into test systems in Testing Farm.
43+
5. Enter `testing-farm request`.
44+
Edit to your needs.
45+
46+
```bash
47+
$ TESTING_FARM_API_TOKEN=<your_api_token> \
48+
testing-farm request --pipeline-type="tmt-multihost" \
49+
--plan-filter="tag:playbooks_parallel" \
50+
--git-url "https://github.com/<my_user>/network" \
51+
--git-ref "<my_branch>" \
52+
--compose CentOS-Stream-9 \
53+
-e "SYSTEM_ROLES_ONLY_TESTS=tests_default.yml" \
54+
--no-wait
55+
```

plans/test_playbooks_parallel.fmf

Lines changed: 20 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -14,48 +14,55 @@ provision:
1414
connection: system
1515
hardware:
1616
network:
17-
- type: eth
18-
- type: eth
17+
- type: eth
18+
- type: eth
1919
- name: managed-node2
2020
role: managed_node
2121
connection: system
2222
hardware:
2323
network:
24-
- type: eth
25-
- type: eth
24+
- type: eth
25+
- type: eth
2626
- name: managed-node3
2727
role: managed_node
2828
connection: system
2929
hardware:
3030
network:
31-
- type: eth
32-
- type: eth
31+
- type: eth
32+
- type: eth
3333
- name: managed-node4
3434
role: managed_node
3535
connection: system
3636
hardware:
3737
network:
38-
- type: eth
39-
- type: eth
38+
- type: eth
39+
- type: eth
4040
- name: managed-node5
4141
role: managed_node
4242
connection: system
4343
hardware:
4444
network:
45-
- type: eth
46-
- type: eth
45+
- type: eth
46+
- type: eth
4747
- name: managed-node6
4848
role: managed_node
4949
connection: system
5050
hardware:
5151
network:
52-
- type: eth
53-
- type: eth
52+
- type: eth
53+
- type: eth
54+
- name: managed-node7
55+
role: managed_node
56+
connection: system
57+
hardware:
58+
network:
59+
- type: eth
60+
- type: eth
5461
environment:
5562
ANSIBLE_VER: 2.17
5663
REPO_NAME: network
5764
PYTHON_VERSION: 3.12
58-
SYSTEM_ROLES_ONLY_TESTS: ""
65+
SYSTEM_ROLES_ONLY_TESTS: "" # tests_default.yml
5966
TEST_LOCAL_CHANGES: true
6067
PR_NUM: ""
6168
LINUXSYSTEMROLES_USER: ""
@@ -82,8 +89,6 @@ prepare:
8289
fi
8390
yum install epel-release yum-utils -y
8491
yum-config-manager --enable epel epel-debuginfo epel-source
85-
- name: Test NICs
86-
script: lshw -C network
8792
discover:
8893
- name: Prepare managed node
8994
how: fmf

0 commit comments

Comments
 (0)