Skip to content

Commit cf3dbdf

Browse files
MattDodsonEnglishMattDodsonEnglish
authored
Fix, reorder and simplify threshold instructions (#1147)
- Move instructions to set syntax to top of page - Move extensive annotation directly to code comments - Simplify the language and make it more scannable
1 parent 026c956 commit cf3dbdf

File tree

1 file changed

+141
-125
lines changed

1 file changed

+141
-125
lines changed

‎src/data/markdown/translated-guides/en/02 Using k6/04 Thresholds.md‎

Lines changed: 141 additions & 125 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,8 @@ Thresholds are also essential for [load-testing automation](/testing-guides/auto
2323

2424
After that, you need to worry about the test only after your SUT fails to meet its performance expectations.
2525

26-
## Example: Set threshold for HTTP errors and response duration
26+
## Example: thresholds for HTTP errors and response duration
27+
2728

2829
This sample script specifies two thresholds.
2930
One threshold evaluates the rate of HTTP errors (`http_req_failed` metric).
@@ -66,8 +67,143 @@ In this case, the test met the criteria for both thresholds.
6667
k6 considers this test a `pass` and exits with an exit code `0`.
6768
6869
If any of the thresholds had failed, the little green checkmark <span style="color:green; font-weight:bold">✓</span> next to the threshold name
69-
(`http_req_failed`, `http_req_duration`) would have been a red cross <span style="color:red; font-weight:bold">✗</span>,
70-
and k6 would have generated a non-zero exit code.
70+
(`http_req_failed`, `http_req_duration`) would be a red cross <span style="color:red; font-weight:bold">✗</span>
71+
and k6 would exit with a non-zero exit code.
72+
73+
## Threshold Syntax
74+
75+
To use a threshold, follow these steps:
76+
77+
1. In the `thresholds` property of the `options` object, set a key using the name of the metric you want the threshold for:
78+
```
79+
export const options = {
80+
thresholds: {
81+
...
82+
}
83+
```
84+
2. Define at least one threshold expression. You can do this in two ways:
85+
- The short format puts all threshold expressions as strings in an array.
86+
- The long format puts each threshold in an object, with extra properties to [abort on failure](#abort).
87+
88+
```
89+
export const options = {
90+
thresholds: {
91+
//short format
92+
METRIC_NAME: ["THRESHOLD_EXPRESSION", `...`],
93+
//long format
94+
METRIC_NAME2: [
95+
{
96+
threshold: "THRESHOLD_EXPRESSION",
97+
abortOnFail: true, // boolean
98+
delayAbortEval: "10s", // string
99+
},
100+
], // full format
101+
},
102+
};
103+
```
104+
105+
Note that `METRIC_NAME1` and `THRESHOLD_EXPRESSION` are placeholders.
106+
The real text must be the name of the metric and the threshold expression.
107+
108+
109+
This declaration configures thresholds for the metrics `metric_name1` and `metric_name2`.
110+
To determine whether the threshold passes or fails, the script evaluates the `'threshold_expression'.`
111+
112+
### Threshold expression syntax
113+
114+
A threshold expression evaluates to `true` or `false`.
115+
The threshold expression must be in the following format:
116+
117+
```
118+
<aggregation_method> <operator> <value>
119+
```
120+
121+
Some examples of threshold expressions are as follows:
122+
123+
- `avg < 200` // average duration must be less than 200ms
124+
- `count >= 500` // count must be larger than or equal to 500
125+
- `p(90) < 300` // 90% of samples must be below 300
126+
127+
128+
### Aggregation methods by type
129+
130+
Each of the four [metric types](/using-k6/metrics/#metric-types) included in k6 provides a set of aggregation methods that you can use in threshold expressions.
131+
132+
| Metric type | Aggregation methods |
133+
|-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
134+
| Counter | `count` and `rate` |
135+
| Gauge | `value` |
136+
| Rate | `rate` |
137+
| Trend | `avg`, `min`, `max`, `med` and `p(N)` where `N` specifies the threshold percentile value, expressed as a number between 0.0 and 100. E.g. `p(99.99)` means the 99.99th percentile. The values are in milliseconds. |
138+
| | |
139+
140+
This (slightly contrived) sample script uses all different types of metrics,
141+
setting different types of thresholds for each:
142+
143+
<CodeGroup labels={["thresholds-all.js"]} lineNumbers={[true]}>
144+
145+
```javascript
146+
import http from 'k6/http';
147+
import { Trend, Rate, Counter, Gauge } from 'k6/metrics';
148+
import { sleep } from 'k6';
149+
150+
export const TrendRTT = new Trend('RTT');
151+
export const RateContentOK = new Rate('Content OK');
152+
export const GaugeContentSize = new Gauge('ContentSize');
153+
export const CounterErrors = new Counter('Errors');
154+
export const options = {
155+
thresholds: {
156+
// Count: Incorrect content cannot be returned more than 99 times.
157+
'Errors': ['count<100'],
158+
// Gauge: returned content must be smaller than 4000 bytes
159+
'ContentSize': ['value<4000'],
160+
// Rate: content must be OK more than 95 times
161+
'Content OK': ['rate>0.95'],
162+
// Trend: Percentiles, averages, medians, and minimums
163+
// must be within specified milliseconds.
164+
'RTT': ['p(99)<300', 'p(70)<250', 'avg<200', 'med<150', 'min<100'],
165+
},
166+
};
167+
168+
export default function () {
169+
const res = http.get('https://test-api.k6.io/public/crocodiles/1/');
170+
const contentOK = res.json('name') === 'Bert';
171+
172+
TrendRTT.add(res.timings.duration);
173+
RateContentOK.add(contentOK);
174+
GaugeContentSize.add(res.body.length);
175+
CounterErrors.add(!contentOK);
176+
177+
sleep(1);
178+
}
179+
```
180+
181+
</CodeGroup>
182+
183+
<Blockquote mod="attention" title="">
184+
185+
Do not specify multiple thresholds for the same metric by repeating the same object key.
186+
187+
</Blockquote>
188+
189+
Since thresholds are defined as the properties of a JavaScript object, you can't specify multiple ones with the same property name.
190+
191+
<CodeGroup labels={["threshold-duplicate-mistake.js"]} lineNumbers={[true]}>
192+
193+
```javascript
194+
export const options = {
195+
thresholds: {
196+
// don't use the same metric more than once here
197+
metric_name: ["count<100"],
198+
metric_name: ["rate<50"],
199+
},
200+
};
201+
```
202+
203+
</CodeGroup>
204+
205+
The rest will be **silently** ignored.
206+
If you want to set multiple thresholds for a metric, specify them with an [array for the same key](/using-k6/thresholds/#multiple-thresholds-on-a-single-metric).
71207
72208
## Threshold examples to copy and paste
73209
@@ -191,127 +327,7 @@ export default function () {
191327
</CodeGroup>
192328
193329
194-
## Threshold Syntax
195-
196-
To use a threshold, define at least one `threshold_expression`:
197330
198-
<CodeGroup labels={["threshold-options.js"]} lineNumbers={[true]}>
199-
200-
```javascript
201-
export const options = {
202-
thresholds: {
203-
metric_name1: ['threshold_expression', `...`], // short format
204-
metric_name2: [
205-
{
206-
threshold: 'threshold_expression',
207-
abortOnFail: true, // boolean
208-
delayAbortEval: '10s', // string
209-
},
210-
], // full format
211-
},
212-
};
213-
```
214-
215-
</CodeGroup>
216-
217-
This declaration configures thresholds for the metrics `metric_name1` and `metric_name2`.
218-
To determine whether the threshold passes or fails, the script evaluates the `'threshold_expression'.`
219-
220-
The threshold expression must be in the following format:
221-
222-
```
223-
<aggregation_method> <operator> <value>
224-
```
225-
226-
For example:
227-
228-
- `avg < 200` // average duration must be less than 200ms
229-
- `count >= 500` // count must be larger than or equal to 500
230-
- `p(90) < 300` // 90% of samples must be below 300
231-
232-
A threshold expression evaluates to `true` or `false`.
233-
234-
Each of the four [metric types](/using-k6/metrics/#metric-types) included in k6 provides a set of aggregation methods that you can use in threshold expressions.
235-
236-
| Metric type | Aggregation methods |
237-
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
238-
| Counter | `count` and `rate` |
239-
| Gauge | `value` |
240-
| Rate | `rate` |
241-
| Trend | `avg`, `min`, `max`, `med` and `p(N)` where `N` is a number between 0.0 and 100.0 meaning the percentile value to look at, e.g. `p(99.99)` means the 99.99th percentile. The unit for these values is milliseconds. |
242-
243-
This (slightly contrived) sample script uses all different types of metrics,
244-
setting different types of thresholds for each:
245-
246-
<CodeGroup labels={["thresholds-all.js"]} lineNumbers={[true]}>
247-
248-
```javascript
249-
import http from 'k6/http';
250-
import { Trend, Rate, Counter, Gauge } from 'k6/metrics';
251-
import { sleep } from 'k6';
252-
253-
export const TrendRTT = new Trend('RTT');
254-
export const RateContentOK = new Rate('Content OK');
255-
export const GaugeContentSize = new Gauge('ContentSize');
256-
export const CounterErrors = new Counter('Errors');
257-
export const options = {
258-
thresholds: {
259-
'Errors': ['count<100'],
260-
'ContentSize': ['value<4000'],
261-
'Content OK': ['rate>0.95'],
262-
'RTT': ['p(99)<300', 'p(70)<250', 'avg<200', 'med<150', 'min<100'],
263-
},
264-
};
265-
266-
export default function () {
267-
const res = http.get('https://test-api.k6.io/public/crocodiles/1/');
268-
const contentOK = res.json('name') === 'Bert';
269-
270-
TrendRTT.add(res.timings.duration);
271-
RateContentOK.add(contentOK);
272-
GaugeContentSize.add(res.body.length);
273-
CounterErrors.add(!contentOK);
274-
275-
sleep(1);
276-
}
277-
```
278-
279-
</CodeGroup>
280-
281-
We have these thresholds:
282-
283-
- A counter metric that keeps track of the total number of times that the content response was *not* `OK`.
284-
The success criteria here is that content cannot be bad more than 99 times.
285-
- A gauge metric that contains the latest size of the returned content.
286-
The success criteria for this metric is that the returned content is smaller than 4000 bytes.
287-
- A rate metric that keeps track of how often the content returned was `OK`. This metric has one
288-
success criteria: content must have been `OK` more than 95% of the time.
289-
- A trend metric that is fed with response time samples and which has the following threshold criteria:
290-
- 99th percentile response time must be below 300 ms
291-
- 70th percentile response time must be below 250 ms
292-
- Average response time must be below 200 ms
293-
- Median response time must be below 150 ms
294-
- Minimum response time must be below 100 ms
295-
296-
**⚠️ Common mistake** Do not specify multiple thresholds for the same metric by repeating the same object key:
297-
298-
<CodeGroup labels={["threshold-duplicate-mistake.js"]} lineNumbers={[true]}>
299-
300-
```javascript
301-
export const options = {
302-
thresholds: {
303-
// avoid using the same metric more than once here
304-
// metric_name: [ 'count<100' ],
305-
// metric_name: [ 'rate<50' ],
306-
},
307-
};
308-
```
309-
</CodeGroup>
310-
311-
Since thresholds are defined as the properties of a JavaScript object, it's not possible to specify multiple ones with the same property name.
312-
Only the last one will remain.
313-
The rest will be **silently** ignored.
314-
Instead, specify them with an [array for the same key](/using-k6/thresholds/#multiple-thresholds-on-a-single-metric).
315331
316332
## Set thresholds for specific tags
317333
@@ -366,7 +382,7 @@ export default function () {
366382
367383
</CodeGroup>
368384
369-
## Abort a test when a threshold is crossed
385+
## Abort a test when a threshold is crossed {#abort}
370386
371387
If you want to abort a test as soon as a threshold is crossed,
372388
set the `abortOnFail` property to `true`.
@@ -433,7 +449,7 @@ Therefore, the `abortOnFail` feature may be delayed by up to 60 seconds.
433449
434450
</Blockquote>
435451
436-
## Failing a load test using checks
452+
## Fail a load test using checks
437453
438454
[Checks](/using-k6/checks) are nice for codifying assertions, but unlike `thresholds`, `checks` do not affect the exit status of k6.
439455

0 commit comments

Comments
 (0)