You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: NEWS.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,6 @@
1
1
# testthat (development version)
2
2
3
+
*`vignette("custom-expectations)` has been overhauled to make it much clearer how to create high-quality expectations (#2113, #2132, #2072).
3
4
*`expect_snapshot()` and friends will now fail when creating a new snapshot on CI. This is usually a signal that you've forgotten to run it locally before committing (#1461).
4
5
*`expect_snapshot_value()` can now handle expressions that generate `-` (#1678) or zero length atomic vectors (#2042).
5
6
*`expect_matches()` failures should be a little easier to read (#2135).
This vignette shows you how to write your expectations that work identically to the built-in `expect_` functions.
20
-
21
-
You can use these either locally by putting them in a helper file, or export them from your package.
19
+
This vignette shows you how to write your own expectations. You can use them within your package by putting them in a helper file, or share them with others by exporting them from your package.
22
20
23
21
## Expectation basics
24
22
25
-
There are three main parts to writing an expectation, as illustrated by `expect_length()`:
23
+
An expectation has three main parts, as illustrated by `expect_length()`:
26
24
27
25
```{r}
28
26
expect_length <- function(object, n) {
29
27
# 1. Capture object and label
30
28
act <- quasi_label(rlang::enquo(object))
31
29
32
-
# 2. Verify the expectations
30
+
# 2. Check if expectations are violated
33
31
act_n <- length(act$val)
34
32
if (act_n != n) {
35
33
msg <- sprintf("%s has length %i, not length %i.", act$lab, act_n, n)
36
34
return(fail(msg))
37
35
}
38
36
39
-
# 3. Pass
37
+
# 3. Pass when expectations are met
40
38
pass(act$val)
41
39
}
42
40
```
43
41
44
-
### Capture valueand label
42
+
The first step in any expectation is to use `quasi_label()` to capture a "labelled value", i.e. a list that contains both the value (`$val`) for testing and a label (`$lab`) for messaging. This is a pattern that exists for fairly esoteric reasons; you don't need to understand it, just copy and paste it 🙂.
45
43
46
-
The first step in any expectation is to capture the actual object, and generate a label for it to use if a failure occur. All testthat expectations support quasiquotation so that you can unquote variables. This makes it easier to generate good labels when the expectation is called from a function or within a for loop.
44
+
Next you need to check each way that `object` could violate the expectation. In this case, there's only one check, but in more complicated cases there can be multiple checks. In most cases, it's easier to check for violations one by one, using early returns to `fail()`. This makes it easier to write informative failure messages that state both what the object is and what you expected.
47
45
48
-
By convention, the first argument to every `expect_` function is called `object`, and you capture its value (`val`) and label (`lab`) with `act <- quasi_label(enquo(object))`, where `act` is short for actual (in constrast to expected).
46
+
Also note that you need to use `return(fail())` here. You won't see the problem when interactively testing your function because when run outside of `test_that()`, `fail()` throws an error, causing the function to terminate early. When running inside of `test_that()`, however, `fail()` does not stop execution because we want to collect all failures in a given test.
49
47
50
-
### Verify the expectation
48
+
Finally, if the object is as expected, call `pass()` with `act$val`. Returning the input value is good practice since expectation functions are called primarily for their side-effects (triggering a failure). This allows expectations to be chained:
51
49
52
-
Now we can check if our expectation is met and return `fail()` if not. The most challenging job here is typically generating the error message because you want it to be as self-contained as possible. This means it should typically give both the expected and actual value, along with the name of the object passed to the expectation. testthat expectations use `sprintf()`, but if you're familiar with {glue}, you might want to use that instead.
50
+
```{r}
51
+
mtcars |>
52
+
expect_type("list") |>
53
+
expect_s3_class("data.frame") |>
54
+
expect_length(11)
55
+
```
56
+
57
+
### Testing your expectations
53
58
54
-
More complicated expectations will have more `if` statements. For example, we might want to make our `expect_length()` function include an assertion that `object` is a vector:
59
+
Once you've written your expectation, you need to test it, and luckily testthat comes with three expectations designed specifically to test expectations:
60
+
61
+
*`expect_success()` checks that your expectation emits exactly one success and zero failures.
62
+
*`expect_failure()` checks that your expectation emits exactly one failure and zero successes.
63
+
*`expect_failure_snapshot()` captures the failure message in a snapshot, making it easier to review if it's useful or not.
64
+
65
+
The first two expectations are particularly important because they ensure that your expectation reports the correct number of successes and failures to the user.
The following sections show you a few more variations, loosely based on existing testthat expectations.
83
+
84
+
### `expect_vector_length()`
85
+
86
+
Let's make `expect_length()` a bit more strict by also checking that the input is a vector. R is a bit weird in that it gives a length to pretty much every object, and you can imagine not wanting this code to succeed:
87
+
88
+
```{r}
89
+
expect_length(mean, 1)
90
+
```
91
+
92
+
To do this we'll add an extra check that the input is either an atomic vector or a list:
Note that it's really important to `return(fail())` here. You wont see the problem when interactively testing your function because when run outside of `test_that()`, `fail()` throws an error, causing the function to terminate early. When running inside of `test_that()` however, `fail()` does not stop execution because we want to collect all failures in a given test.
115
+
```{r}
116
+
#| error: true
117
+
expect_vector_length(mean, 1)
118
+
expect_vector_length(mtcars, 15)
119
+
```
76
120
77
-
### Pass the test
121
+
### `expect_s3_class()`
78
122
79
-
If no assertions fail, call `pass()` with the input value (usually `act$val`). Returning the input value is good practice since expectation functions are called primarily for their side-effects (triggering a failure). This allows expectations to be chained:
123
+
Or imagine if you're checking to see if an object inherits from an S3 class. In R, there's no direct way to tell if an object is an S3 object: you can confirm that it's an object, then that it's not an S4 object. So you might organize your expectation this way:
80
124
81
125
```{r}
82
-
mtcars |>
83
-
expect_type("list") |>
84
-
expect_s3_class("data.frame") |>
85
-
expect_length(11)
126
+
expect_s3_class <- function(object, class) {
127
+
if (!rlang::is_string(class)) {
128
+
rlang::abort("`class` must be a string.")
129
+
}
130
+
131
+
act <- quasi_label(rlang::enquo(object))
132
+
133
+
if (!is.object(act$val)) {
134
+
return(fail(sprintf("%s is not an object.", act$lab)))
135
+
}
136
+
137
+
if (isS4(act$val)) {
138
+
return(fail(sprintf("%s is an S4 object, not an S3 object.", act$lab)))
testthat comes with three expectations designed specifically to test expectations: `expect_success()` and `expect_failure()`:
168
+
Note that I also check that the `class` argument must be a string. This is an error, not a failure, because it suggests you're using the function incorrectly.
91
169
92
-
*`expect_success()` checks that your expectation emits exactly one success and zero failures.
93
-
*`expect_failure()` checks that your expectation emits exactly one failure and zero successes.
94
-
*`expect_failure_snapshot()` captures the failure message in a snapshot, making it easier to review if it's useful or not.
170
+
```{r}
171
+
#| error: true
172
+
expect_s3_class(x1, 1)
173
+
```
174
+
175
+
## Repeated code
176
+
177
+
As you write more expectations, you might discover repeated code that you want to extract out into a helper. Unfortunately, creating helper functions is not straightforward in testthat because every `fail()` captures the calling environment in order to give maximally useful tracebacks. Because getting this right is not critical (you'll just get a slightly suboptimal traceback in the case of failure), we don't recommend bothering. However, we document it here because it's important to get it right in testthat itself.
178
+
179
+
The key challenge is that `fail()` captures a `trace_env` which should be the execution environment of the expectation. This usually works, because the default value of `trace_env` is `caller_env()`. But when you introduce a helper, you'll need to explicitly pass it along:
95
180
96
181
```{r}
97
-
test_that("expect_length works as expected", {
98
-
x <- 1:10
99
-
expect_success(expect_length(x, 10))
100
-
expect_failure(expect_length(x, 11))
182
+
expect_length_ <- function(act, n, trace_env = caller_env()) {
183
+
act_n <- length(act$val)
184
+
if (act_n != n) {
185
+
msg <- sprintf("%s has length %i, not length %i.", act$lab, act_n, n)
186
+
return(fail(msg, trace_env = trace_env))
187
+
}
101
188
102
-
expect_snapshot_failure(expect_length(x, 11))
103
-
})
189
+
pass(act$val)
190
+
}
191
+
192
+
expect_length <- function(object, n) {
193
+
act <- quasi_label(rlang::enquo(object))
194
+
expect_length_(act, n)
195
+
}
104
196
```
197
+
198
+
A few recommendations:
199
+
200
+
* The helper shouldn't be user facing, so we give it a `_` suffix to make that clear.
201
+
* It's typically easiest for a helper to take the labelled value produced by `quasi_label()`.
202
+
* Your helper should usually call both `fail()` and `pass()` and be returned from the wrapping expectation.
0 commit comments