@@ -67,9 +67,54 @@ then run
67
67
... as root.
68
68
69
69
70
- Running the System Tests
70
+ Running the System Tests with pytest
71
71
===
72
72
73
+ The pytest system test runner is currently in development, but it is the
74
+ recommended way to run tests. Please report issues to QA.
75
+
76
+ Running an Individual Test
77
+ ---
78
+
79
+ pytest -k <test-name>
80
+
81
+ Note that in comparison to the legacy test runner, some additional tests might
82
+ be picked up when specifying just the system test directory name. To check
83
+ which tests will be executed, you can use the `--collect-only` option. You
84
+ might also be able to find a more specific test name to provide to ensure only
85
+ your desired test is executed. See help for `-k` option in `pytest --help` for
86
+ more info.
87
+
88
+ It is also possible to run a single individual pytest test case. For example,
89
+ you can use the name test_sslyze_dot to execute just the test_sslyze_dot()
90
+ function from doth/tests_sslyze.py. The entire needed setup and teardown will
91
+ be handled by the framework.
92
+
93
+ Running All the System Tests
94
+ ---
95
+
96
+ Issuing plain `pytest` command without any argument will execute all tests
97
+ sequenatially. To execute them in parallel, ensure you have pytest-xdist
98
+ installed and run:
99
+
100
+ pytest --dist loadscope -n <number-of-workers>
101
+
102
+ It is vital to provide the `--dist loadscope` option when running the tests in
103
+ parallel to ensure tests from a single module are executed from the same
104
+ thread. Otherwise, there's a risk of port contention and inefficient use of
105
+ resources.
106
+
107
+
108
+ Running the System Tests Using the Legacy Runner
109
+ ===
110
+
111
+ !!! WARNING !!!
112
+ ---
113
+ The legacy way to run system tests is currently being reworked into a pytest
114
+ system test runner described in the previous section. The contents of this
115
+ section might be out of date and no longer applicable. Please try and use the
116
+ pytest runner if possible and report issues and missing features.
117
+
73
118
Running an Individual Test
74
119
---
75
120
The tests can be run individually using the following command:
0 commit comments