Skip to content

Commit b82952b

Browse files
committed
Add guide for define_test()
1 parent 8c1e9fa commit b82952b

File tree

1 file changed

+108
-1
lines changed

1 file changed

+108
-1
lines changed

docs/+current/modules/testing/cmake/03_test.md

Lines changed: 108 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,111 @@ author: RSP Systems A/S
77

88
# Test
99

10-
_TODO: ..._
10+
The `define_test()` function is used to describe what callback (_function_) must be invoked when tests are executed.
11+
Behind the scene, this function is responsible to
12+
[register](https://cmake.org/cmake/help/latest/command/add_test.html#command:add_test) the test for ctest.
13+
This is done by adding a ctest that invokes an "intermediary" - a [test executor](./05_executor.md) - which is
14+
responsible for invoking the specified callback, via `define_test()`.
15+
16+
[TOC]
17+
18+
## Parameters
19+
20+
* < name >: _Human readable name of test case._
21+
* < callback >: _The function that contains the actual test logic._
22+
* `DATA_PROVIDER`: (_optional_), _Command or macro that provides data-set(s) for the test. See [data providers](#data-providers) for details._
23+
* `EXPECT_FAILURE`: (_optional_), _Options, if specified then callback is expected to fail. See [failure expectations](#failure-expectations) for details._
24+
* `SKIP`: (_optional_), _Option, if set then test will be marked as "disabled" and not executed. See [skipping tests](#skipping-tests) for details._
25+
26+
!!! warning "Rebuild Required"
27+
Changes to function `define_test()` parameters requires you to rebuild your project, before the changes take effect.
28+
29+
!!! danger "Caution"
30+
Although the `< callback >` parameter can accept a
31+
[marco](https://cmake.org/cmake/help/latest/command/macro.html#macro), you _SHOULD_ always use a
32+
[function](https://cmake.org/cmake/help/latest/command/function.html#command:function) for defining the actual
33+
test logic. Using a marco can lead to undesired side effects. Please read CMake's
34+
["Macro vs. Function"](https://cmake.org/cmake/help/latest/command/macro.html#macro-vs-function) for additional
35+
details.
36+
37+
## Example
38+
39+
```cmake
40+
include("rsp/testing")
41+
42+
# ... previous not shown ...
43+
44+
define_test("assets are ready after build" "asserts_ready")
45+
function(asserts_ready)
46+
47+
# ...actual test logic not shown here...
48+
49+
assert_truthy(assets_exist MESSAGE "Assets have not been built...")
50+
endfunction()
51+
```
52+
53+
## Failure Expectations
54+
55+
When you need to test logic that is intended to fail when certain conditions are true, then you can mark your test
56+
to expect a failure. This is done by setting the `EXPECT_FAILURE` option.
57+
58+
```cmake
59+
define_test("fails when assets not built" "fails_when_not_ready" EXPECT_FAILURE)
60+
function(fails_when_not_ready)
61+
62+
# ...actual test logic not shown here...
63+
64+
assert_truthy(false)
65+
endfunction()
66+
```
67+
68+
Behind the scene, ctest's [`WILL_FAIL`](https://cmake.org/cmake/help/latest/prop_test/WILL_FAIL.html#prop_test:WILL_FAIL)
69+
property is set for the given test, when the `EXPECT_FAILURE` option is set.
70+
71+
## Data Providers
72+
73+
You can specify a function or marco as a test's data-provider, via the `DATA_PROVIDER` parameter.
74+
Doing so will result in the same test being executed multiple times, with different sets of data.
75+
The specified function or marco **MUST** assign a list of "items" (_test data_) to the given `< output >` variable.
76+
77+
```cmake
78+
function(provides_data output)
79+
set("${output}" "a;b;c;d")
80+
return (PROPAGATE "${output}")
81+
endfunction()
82+
83+
define_test(
84+
"My Test"
85+
"my_test"
86+
DATA_PROVIDER "provides_data"
87+
)
88+
function(my_test letter)
89+
string(LENGTH "${letter}" length)
90+
91+
assert_greater_than(0 length MESSAGE "No letter provided: (length: ${length})")
92+
endfunction()
93+
```
94+
95+
In the above "My Test" will be registered multiple times, one for each item provided by the `provides_data()` function.
96+
Each data-set item is then passed on to the test, as an argument.
97+
98+
!!! warning "Rebuild Required"
99+
Whenever you change the items provided by a "data provider" function, you will be required to rebuild
100+
your CMake project, before the changes are reflected by the executed tests!
101+
102+
## Skipping tests
103+
104+
Set the `SKIP` option, if you wish to ensure that a test is not executed by ctest.
105+
106+
```cmake
107+
# Test is SKIPPED
108+
define_test("can build with bitmap pictures" "can_build_with_bitmap" SKIP)
109+
function(can_build_with_bitmap)
110+
111+
# ...not shown...
112+
113+
endfunction()
114+
```
115+
116+
Behind the scene, ctest's [`DISABLED`](https://cmake.org/cmake/help/latest/prop_test/DISABLED.html)
117+
property is set, when a test is marked as skipped.

0 commit comments

Comments
 (0)