Skip to content
This repository was archived by the owner on Aug 13, 2025. It is now read-only.

Commit 4242586

Browse files
committed
Update user docs with pytest system test runner
1 parent b39b65b commit 4242586

File tree

1 file changed

+46
-1
lines changed

1 file changed

+46
-1
lines changed

bin/tests/system/README

Lines changed: 46 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,9 +67,54 @@ then run
6767
... as root.
6868

6969

70-
Running the System Tests
70+
Running the System Tests with pytest
7171
===
7272

73+
The pytest system test runner is currently in development, but it is the
74+
recommended way to run tests. Please report issues to QA.
75+
76+
Running an Individual Test
77+
---
78+
79+
pytest -k <test-name>
80+
81+
Note that in comparison to the legacy test runner, some additional tests might
82+
be picked up when specifying just the system test directory name. To check
83+
which tests will be executed, you can use the `--collect-only` option. You
84+
might also be able to find a more specific test name to provide to ensure only
85+
your desired test is executed. See help for `-k` option in `pytest --help` for
86+
more info.
87+
88+
It is also possible to run a single individual pytest test case. For example,
89+
you can use the name test_sslyze_dot to execute just the test_sslyze_dot()
90+
function from doth/tests_sslyze.py. The entire needed setup and teardown will
91+
be handled by the framework.
92+
93+
Running All the System Tests
94+
---
95+
96+
Issuing plain `pytest` command without any argument will execute all tests
97+
sequenatially. To execute them in parallel, ensure you have pytest-xdist
98+
installed and run:
99+
100+
pytest --dist loadscope -n <number-of-workers>
101+
102+
It is vital to provide the `--dist loadscope` option when running the tests in
103+
parallel to ensure tests from a single module are executed from the same
104+
thread. Otherwise, there's a risk of port contention and inefficient use of
105+
resources.
106+
107+
108+
Running the System Tests Using the Legacy Runner
109+
===
110+
111+
!!! WARNING !!!
112+
---
113+
The legacy way to run system tests is currently being reworked into a pytest
114+
system test runner described in the previous section. The contents of this
115+
section might be out of date and no longer applicable. Please try and use the
116+
pytest runner if possible and report issues and missing features.
117+
73118
Running an Individual Test
74119
---
75120
The tests can be run individually using the following command:

0 commit comments

Comments
 (0)