1
1
Introduction
2
2
============
3
3
``pytest-order `` is a pytest plugin which allows you to customize the order
4
- in which run your tests are run. It provides the marker ``order ``, that has
4
+ in which your tests are run. It provides the marker ``order ``, that has
5
5
attributes that defines when your tests should run in relation to each other.
6
6
These attributes can be absolute (i.e. first, or second-to-last) or relative
7
7
(i.e. run this test before this other test).
@@ -28,10 +28,14 @@ Here are examples for which markers correspond to markers in
28
28
29
29
Supported Python and pytest versions
30
30
------------------------------------
31
- pytest-order supports python 2.7, 3.5 - 3.9, and pypy/pypy3, and is
31
+ `` pytest-order `` supports python 2.7, 3.5 - 3.9, and pypy/pypy3, and is
32
32
compatible with pytest 3.6.0 or newer. Note that support for Python 2 will
33
33
be removed in one of the next versions.
34
34
35
+ All supported combinations of Python and pytest versions are tested in
36
+ the CI builds. While these tests run under Linux, the plugin shall work
37
+ under MacOs and Windows as well.
38
+
35
39
Installation
36
40
------------
37
41
The latest released version can be installed from
@@ -60,7 +64,7 @@ For example, for the following tests:
60
64
def test_bar ():
61
65
assert True
62
66
63
- Here is the output :
67
+ the output is something like :
64
68
65
69
::
66
70
@@ -74,7 +78,7 @@ Here is the output:
74
78
75
79
=========================== 2 passed in 0.01 seconds ===========================
76
80
77
- With pytest-order, you can change the default ordering as follows:
81
+ With `` pytest-order `` , you can change the default ordering as follows:
78
82
79
83
.. code :: python
80
84
@@ -88,6 +92,8 @@ With pytest-order, you can change the default ordering as follows:
88
92
def test_bar ():
89
93
assert True
90
94
95
+ This will generate the output:
96
+
91
97
::
92
98
93
99
$ py.test test_foo.py -vv
@@ -104,6 +110,13 @@ With pytest-order, you can change the default ordering as follows:
104
110
Usage
105
111
=====
106
112
The above is a trivial example, but ordering is respected across test files.
113
+
114
+ .. note ::
115
+ The scope of the ordering is always global, e.g. tests with lower ordinal
116
+ numbers are always executed before tests with higher numbers, regardless of
117
+ the module and class they reside in. This may be changed to be
118
+ configurable in a later version.
119
+
107
120
There are currently three possibilities to define the order:
108
121
109
122
Order by number
@@ -152,9 +165,9 @@ There is no limit for the numbers that can be used in this way.
152
165
Order using ordinals
153
166
--------------------
154
167
155
- You can also use markers such as "first", "second", "last", and
156
- "second_to_last". These are convenience notations, and have the same effect
157
- as the numbers 0, 1, -1 and -2 that have been shown above:
168
+ Instead of the numbers, you can use ordinal names such as "first", "second",
169
+ "last", and " second_to_last". These are convenience notations, and have the
170
+ same effect as the numbers 0, 1, -1 and -2 that have been shown above:
158
171
159
172
.. code :: python
160
173
@@ -191,8 +204,8 @@ as the numbers 0, 1, -1 and -2 that have been shown above:
191
204
192
205
=========================== 4 passed in 0.02 seconds ===========================
193
206
194
- Convenience names are only defined for the first and the last 8 numbers,
195
- here is the complete list with the corresponding numbers:
207
+ Convenience names are only defined for the first and the last 8 numbers.
208
+ Here is the complete list with the corresponding numbers:
196
209
197
210
- 'first': 0
198
211
- 'second': 1
@@ -245,6 +258,12 @@ by their name:
245
258
246
259
=========================== 4 passed in 0.02 seconds ===========================
247
260
261
+ .. note ::
262
+ The `pytest-dependency <https://pypi.org/project/pytest-dependency/ >`__
263
+ plugin also manages dependencies between tests (skips tests that depend
264
+ on skipped or failed tests), but doesn't do any ordering. You can combine
265
+ both plugins if you need both options.
266
+
248
267
Configuration
249
268
=============
250
269
Currently there is only one option that changes the behavior of the plugin.
@@ -255,18 +274,19 @@ You may sometimes find that you want to suggest an ordering of tests, while
255
274
allowing it to be overridden for good reason. For example, if you run your test
256
275
suite in parallel and have a number of tests which are particularly slow, it
257
276
might be desirable to start those tests running first, in order to optimize
258
- your completion time. You can use the pytest-order plugin to inform pytest
277
+ your completion time. You can use the `` pytest-order `` plugin to inform pytest
259
278
of this.
279
+
260
280
Now suppose you also want to prioritize tests which failed during the
261
281
previous run, by using the ``--failed-first `` option. By default,
262
282
pytest-order will override the ``--failed-first `` order, but by adding the
263
283
``--indulgent-ordering `` option, you can ask pytest to run the sort from
264
284
pytest-order *before * the sort from ``--failed-first ``, allowing the failed
265
- tests to be sorted to the front.
285
+ tests to be sorted to the front (note that in pytest versions from 6.0 on,
286
+ this seems not to be needed anymore, at least in this specific case).
266
287
267
288
268
289
.. toctree ::
269
290
:maxdepth: 2
270
291
271
292
.. _markers : https://pytest.org/latest/mark.html
272
-
0 commit comments