|
| 1 | +# launch_pytest |
| 2 | + |
| 3 | +This tool is a framework for launch integration testing. For example: |
| 4 | + |
| 5 | + * The exit codes of all processes are available to the tests. |
| 6 | + * Tests can check that all processes shut down normally, or with specific exit codes. |
| 7 | + * Tests can fail when a process dies unexpectedly. |
| 8 | + * The stdout and stderr of all processes are available to the tests. |
| 9 | + * The command-line used to launch the processes are available to the tests. |
| 10 | + * Some tests run concurrently with the launch and can interact with the running processes. |
| 11 | + |
| 12 | +## Differences with launch_testing |
| 13 | + |
| 14 | +launch_testing is an standalone testing tool, which lacks many features: |
| 15 | + * It's impossible to filter test cases by name and run only some. |
| 16 | + * It's impossible to mark a test as skipped or xfail. |
| 17 | + * The error reporting of the tool was custom, and the output wasn't as nice as the output |
| 18 | + generated by other testing frameworks as unittest and pytest. |
| 19 | + |
| 20 | +launch_pytest is a really simple pytest plugin leveraging pytest fixtures to manage a launch service lifetime easily. |
| 21 | + |
| 22 | +## Quick start example |
| 23 | + |
| 24 | +Start with the [`pytest_hello_world.py`](test/launch_pytest/examples/pytest_hello_world.py) example. |
| 25 | + |
| 26 | +Run the example by doing: |
| 27 | + |
| 28 | +```sh |
| 29 | +python3 -m pytest test/launch_pytest/examples/pytest_hello_world.py |
| 30 | +``` |
| 31 | + |
| 32 | +The `launch_pytest` plugin will launch the nodes found in the `launch_descripture` fixture, run the tests from the `test_read_stdout()` class, shut down the launched nodes, and then run the statements after the `yield` statement in `test_read_stdout()`. |
| 33 | + |
| 34 | +#### launch_pytest fixtures |
| 35 | + |
| 36 | +```python |
| 37 | +@launch_pytest.fixture |
| 38 | +def launch_description(hello_world_proc): |
| 39 | + """Launch a simple process to print 'hello_world'.""" |
| 40 | + return launch.LaunchDescription([ |
| 41 | + hello_world_proc, |
| 42 | + # Tell launch when to start the test |
| 43 | + # If no ReadyToTest action is added, one will be appended automatically. |
| 44 | + launch_pytest.actions.ReadyToTest() |
| 45 | + ]) |
| 46 | +``` |
| 47 | + |
| 48 | +A `@launch_pytest.fixture` function should return a `launch.LaunchDescription` object, or a sequence of objects which first item is a `launch.LaunchDescription`. |
| 49 | +This launch description will be used in all tests with a mark `@pytest.mark.launch(fixture=<your_fixture_name>)`, in this case `<your_fixture_name>=launch_description`. |
| 50 | + |
| 51 | +The launch description can include a `ReadyToTest` action to signal to the test framework that it's safe to start the active tests. |
| 52 | +If one isn't included, a `ReadyToTest` action will be appended at the end. |
| 53 | + |
| 54 | +launch_pytest fixtures can have `module`, `class` or `function` scope, the default is `function`. |
| 55 | +For example: |
| 56 | + |
| 57 | +```python |
| 58 | +@launch_pytest.fixture(scope=my_scope) |
| 59 | +def my_fixture(): |
| 60 | + return LaunchDescription(...) |
| 61 | + |
| 62 | +@pytest.mark.launch(fixture=my_fixture) |
| 63 | +def test_case_1(): |
| 64 | + pass |
| 65 | + |
| 66 | +@pytest.mark.launch(fixture=my_fixture) |
| 67 | +def test_case_2(): |
| 68 | + pass |
| 69 | +``` |
| 70 | + |
| 71 | +If `my_scope=function`, the following happens: |
| 72 | + |
| 73 | +- A launch service using the `LaunchDescription` returned by `my_fixture()` is started. |
| 74 | +- `test_case_1()` is run. |
| 75 | +- The launch service is shutdown. |
| 76 | +- Another launch service using the `LaunchDescription` returned by `my_fixture()` is started, `my_fixture()` is called again. |
| 77 | +- `test_case_2()` is run. |
| 78 | +- The launch service is shutdown. |
| 79 | + |
| 80 | +Whereas when `my_scope=module`, `test case_2()` will run immediately after `test case_1()`, concurrently with the same launch service. |
| 81 | + |
| 82 | +It's not recommended to mix fixtures with `module` scope with fixtures of `class`/`function` scope in the same file. |
| 83 | +It's not recommended to use fixtures with scope larger than `module`. |
| 84 | +A test shouldn't depend on more than one `launch_pytest` fixture. |
| 85 | +Neither of the three things above automatically generates an error in the current `launch_pytest` implementation, but future versions might. |
| 86 | + |
| 87 | +#### Active Tests and shutdwon tests |
| 88 | + |
| 89 | +Test cases marked with `@pytest.mark.launch` will be run concurrently with the launch service or after launch shutdown, depending on the object being marked and the mark arguements. |
| 90 | + |
| 91 | +- functions: Functions marked with `@pytest.mark.launch` will run concurrently with the launch service, except when `shutdown=True` is passed as an argument to the decorator. |
| 92 | + |
| 93 | +```python |
| 94 | +@pytest.mark.launch(fixture=my_ld_fixture) |
| 95 | +def normal_test_case(): |
| 96 | + pass |
| 97 | + |
| 98 | +@pytest.mark.launch(fixture=my_ld_fixture, shutdown=True) |
| 99 | +def shutdown_test_case(): |
| 100 | + pass |
| 101 | +``` |
| 102 | + |
| 103 | +- coroutine functions: The same rules as normal functions apply. |
| 104 | + Coroutines will be run in the same event loop as the launch description, whereas normal functions run concurrently in another thread. |
| 105 | + |
| 106 | +```python |
| 107 | +@pytest.mark.launch(fixture=my_ld_fixture) |
| 108 | +async def normal_test_case(): |
| 109 | + pass |
| 110 | + |
| 111 | +@pytest.mark.launch(fixture=my_ld_fixture, shutdown=True) |
| 112 | +async def shutdown_test_case(): |
| 113 | + pass |
| 114 | +``` |
| 115 | + |
| 116 | +- generators: The first time the generator is called it runs concurrently with the launch service. |
| 117 | + The generator will be resumed after the launch service is shutdown. |
| 118 | + i.e. This allows to write a test that has a step that runs concurrently with the service and one |
| 119 | + that runs after shutdown easily. |
| 120 | + The yielded value is ignored. |
| 121 | + If the generator doesn't stop iteration after being resumed for a second time, the test will fail. |
| 122 | + Passing a `shutdown` argument to the decorator is not allowed in this case. |
| 123 | + |
| 124 | +```python |
| 125 | +@pytest.mark.launch(fixture=my_ld_fixture) |
| 126 | +def normal_test_case(): |
| 127 | + assert True |
| 128 | + yield |
| 129 | + assert True |
| 130 | +``` |
| 131 | + |
| 132 | +- async generators: The same rules as for generators apply here as well. |
| 133 | + The only difference between the two is that async generator will run in the same event loop as the launch service, whereas a generator will run concurrently in another thread. |
| 134 | + |
| 135 | +```python |
| 136 | +@pytest.mark.launch(fixture=my_ld_fixture) |
| 137 | +async def normal_test_case(): |
| 138 | + assert True |
| 139 | + yield |
| 140 | + assert True |
| 141 | +``` |
| 142 | + |
| 143 | +## Fixtures |
| 144 | + |
| 145 | +The `launch_pytest` plugin will provide the following fixtures. |
| 146 | + |
| 147 | +- launch_service: The launch service being used to run the tests. |
| 148 | + It will have the same scope as the launch_pytest fixture with wider scope in the module. |
| 149 | +- launch_context: The launch context being used to run the tests. |
| 150 | + It will have the same scope as the launch_pytest fixture with wider scope in the module. |
| 151 | +- event_loop: The event loop being used to run the launch service and to run async tests. |
| 152 | + It will have the same scope as the launch_pytest fixture with wider scope in the module. |
0 commit comments