Skip to content

Commit b50c038

Browse files
Pepe Canosimskij
Pepe Cano
and
simskij
authored
Add docs for http_req_failed (#237)
* Add `expected_response` tag to HTTP Requests guide * Add `http_req_failed` built-in metric to Metrics guide * Thresholds guide: add threshold examples for `http_req_failed` * Update src/data/markdown/docs/01 guides/02 Using k6/04 Thresholds.md Co-authored-by: Simon Aronsson <[email protected]> * Update src/data/markdown/docs/01 guides/02 Using k6/04 Thresholds.md Co-authored-by: Simon Aronsson <[email protected]> Co-authored-by: Simon Aronsson <[email protected]>
1 parent 15818b5 commit b50c038

File tree

3 files changed

+76
-51
lines changed

3 files changed

+76
-51
lines changed

‎src/data/markdown/docs/01 guides/02 Using k6/01 HTTP requests.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,8 +67,11 @@ k6 will automatically apply [tags](/using-k6/tags-and-groups#section-tags) to yo
6767

6868
| Name | Description |
6969
| ------ | ------------------------------------------ |
70+
| expected_response <sup>(≥ v0.31)</sup> | By default, response statuses between 200 and 399 are `true`. Change the default behavior with [setResponseCallback](/javascript-api/k6-http/setresponsecallback-callback). |
71+
| group | When the request runs inside a [group](/javascript-api/k6/group-name-fn), the tag value is the group name. Default is empty. |
7072
| name | Defaults to URL requested |
7173
| method | Request method (`GET`, `POST`, `PUT` etc.) |
74+
| scenario | When the request runs inside a [scenario](/using-k6/scenarios), the tag value is the scenario name. Default is empty. |
7275
| status | response status |
7376
| url | defaults to URL requested |
7477

@@ -84,9 +87,11 @@ Below you can see how a test result data point (the duration of an HTTP request)
8487
"time": "2017-06-02T23:10:29.52444541+02:00",
8588
"value": 586.831127,
8689
"tags": {
90+
"expected_response": "true",
8791
"group": "",
8892
"method": "GET",
8993
"name": "http://test.k6.io",
94+
"scenario": "",
9095
"status": "200",
9196
"url": "http://test.k6.io"
9297
}
@@ -144,7 +149,6 @@ Which would produce JSON output like the following:
144149
"time":"2017-06-02T23:10:29.52444541+02:00",
145150
"value":586.831127,
146151
"tags": {
147-
"group":"",
148152
"method":"GET",
149153
"name":"PostsItemURL",
150154
"status":"200",
@@ -162,7 +166,6 @@ Which would produce JSON output like the following:
162166
"time":"2017-06-02T23:10:29.58582529+02:00",
163167
"value":580.839273,
164168
"tags": {
165-
"group":"",
166169
"method":"GET",
167170
"name":"PostsItemURL",
168171
"status":"200",
@@ -176,7 +179,7 @@ Which would produce JSON output like the following:
176179

177180
Note how the `name` is the same for the two data samples related to two different URLs. Filtering the results on tag `name: PostsItemURL` will give you a result set including all the data points from all the 100 different URLs.
178181

179-
Additionally, you can use the `http.url` (v0.16.0) wrapper to set the name tag with a string template value:
182+
Additionally, you can use the `http.url` wrapper to set the name tag with a string template value:
180183

181184
<CodeGroup labels={[ ]} lineNumbers={[true]}>
182185

‎src/data/markdown/docs/01 guides/02 Using k6/02 Metrics.md‎

Lines changed: 28 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -17,29 +17,30 @@ The following _built-in_ metrics will **always** be collected by k6:
1717

1818
| Metric Name | Type | Description |
1919
| -------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
20-
| `vus` | Gauge | Current number of active virtual users |
21-
| `vus_max` | Gauge | Max possible number of virtual users (VU resources are pre-allocated, to ensure performance will not be affected when scaling up the load level) |
22-
| `iterations` | Counter | The aggregate number of times the VUs in the test have executed the JS script (the `default` function). |
23-
| `iteration_duration` | Trend | The time it took to complete one full iteration of the default/main function. |
24-
| `dropped_iterations` | Counter | Introduced in k6 v0.27.0, the number of iterations that could not be started due to lack of VUs (for the arrival-rate executors) or lack of time (due to expired maxDuration in the iteration-based executors). |
25-
| `data_received` | Counter | The amount of received data. Read [this example](/examples/track-transmitted-data-per-url) to track data for an individual URL. |
26-
| `data_sent` | Counter | The amount of data sent. Read [this example](/examples/track-transmitted-data-per-url) to track data for an individual URL. |
27-
| `checks` | Rate | The rate of successful checks. |
20+
| vus | Gauge | Current number of active virtual users |
21+
| vus_max | Gauge | Max possible number of virtual users (VU resources are pre-allocated, to ensure performance will not be affected when scaling up the load level) |
22+
| iterations | Counter | The aggregate number of times the VUs in the test have executed the JS script (the `default` function). |
23+
| iteration_duration | Trend | The time it took to complete one full iteration of the default/main function. |
24+
| dropped_iterations | Counter | Introduced in k6 v0.27.0, the number of iterations that could not be started due to lack of VUs (for the arrival-rate executors) or lack of time (due to expired maxDuration in the iteration-based executors). |
25+
| data_received | Counter | The amount of received data. Read [this example](/examples/track-transmitted-data-per-url) to track data for an individual URL. |
26+
| data_sent | Counter | The amount of data sent. Read [this example](/examples/track-transmitted-data-per-url) to track data for an individual URL. |
27+
| checks | Rate | The rate of successful checks. |
2828

2929
## HTTP-specific built-in metrics
3030

3131
_built-in_ metrics will only be generated when/if HTTP requests are made:
3232

3333
| Metric Name | Type | Description |
3434
| -------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
35-
| `http_reqs` | Counter | How many HTTP requests has k6 generated, in total. |
36-
| `http_req_blocked` | Trend | Time spent blocked (waiting for a free TCP connection slot) before initiating the request. `float` |
37-
| `http_req_connecting` | Trend | Time spent establishing TCP connection to the remote host. `float` |
38-
| `http_req_tls_handshaking` | Trend | Time spent handshaking TLS session with remote host |
39-
| `http_req_sending` | Trend | Time spent sending data to the remote host. `float` |
40-
| `http_req_waiting` | Trend | Time spent waiting for response from remote host (a.k.a. \"time to first byte\", or \"TTFB\"). `float` |
41-
| `http_req_receiving` | Trend | Time spent receiving response data from the remote host. `float` |
42-
| `http_req_duration` | Trend | Total time for the request. It's equal to `http_req_sending + http_req_waiting + http_req_receiving` (i.e. how long did the remote server take to process the request and respond, without the initial DNS lookup/connection times). `float` |
35+
| http_reqs | Counter | How many HTTP requests has k6 generated, in total. |
36+
| http_req_blocked | Trend | Time spent blocked (waiting for a free TCP connection slot) before initiating the request. `float` |
37+
| http_req_connecting | Trend | Time spent establishing TCP connection to the remote host. `float` |
38+
| http_req_tls_handshaking | Trend | Time spent handshaking TLS session with remote host |
39+
| http_req_sending | Trend | Time spent sending data to the remote host. `float` |
40+
| http_req_waiting | Trend | Time spent waiting for response from remote host (a.k.a. \"time to first byte\", or \"TTFB\"). `float` |
41+
| http_req_receiving | Trend | Time spent receiving response data from the remote host. `float` |
42+
| http_req_duration | Trend | Total time for the request. It's equal to `http_req_sending + http_req_waiting + http_req_receiving` (i.e. how long did the remote server take to process the request and respond, without the initial DNS lookup/connection times). `float` |
43+
| http_req_failed <sup>(≥ v0.31)</sup> | Rate | The rate of failed requests according to [setResponseCallback](/javascript-api/k6-http/setresponsecallback-callback). |
4344

4445
### Accessing HTTP timings from a script
4546

@@ -61,17 +62,17 @@ In the above snippet, `res` is an [HTTP Response](/javascript-api/k6-http/respon
6162

6263
| Property | Description |
6364
| ----------------------------- | --------------------------------------------------------------------- |
64-
| `res.body` | `string` containing the HTTP response body |
65-
| `res.headers` | `object` containing header-name/header-value pairs |
66-
| `res.status` | `integer` containing HTTP response code received from server |
67-
| `res.timings` | `object` containing HTTP timing information for the request in **ms** |
68-
| `res.timings.blocked` | = `http_req_blocked` |
69-
| `res.timings.connecting` | = `http_req_connecting` |
70-
| `res.timings.tls_handshaking` | = `http_req_tls_handshaking` |
71-
| `res.timings.sending` | = `http_req_sending` |
72-
| `res.timings.waiting` | = `http_req_waiting` |
73-
| `res.timings.receiving` | = `http_req_receiving` |
74-
| `res.timings.duration` | = `http_req_duration` |
65+
| res.body | `string` containing the HTTP response body |
66+
| res.headers | `object` containing header-name/header-value pairs |
67+
| res.status | `integer` containing HTTP response code received from server |
68+
| res.timings | `object` containing HTTP timing information for the request in **ms** |
69+
| res.timings.blocked | = `http_req_blocked` |
70+
| res.timings.connecting | = `http_req_connecting` |
71+
| res.timings.tls_handshaking | = `http_req_tls_handshaking` |
72+
| res.timings.sending | = `http_req_sending` |
73+
| res.timings.waiting | = `http_req_waiting` |
74+
| res.timings.receiving | = `http_req_receiving` |
75+
| res.timings.duration | = `http_req_duration` |
7576

7677
## Custom metrics
7778

‎src/data/markdown/docs/01 guides/02 Using k6/04 Thresholds.md‎

Lines changed: 42 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: 'Thresholds'
3-
excerpt: ''
3+
excerpt: 'Thresholds are a pass/fail criteria used to specify the performance expectations of the system under test.'
44
---
55

66
## What are thresholds?
@@ -9,56 +9,55 @@ Thresholds are a pass/fail criteria used to specify the performance expectations
99

1010
Example expectations (Thresholds):
1111

12-
- System doesn't produce more than 1% errors
12+
- System doesn't produce more than 1% errors.
1313
- Response time for 95% of requests should be below 200ms.
1414
- Response time for 99% of requests should be below 400ms.
1515
- Specific endpoint must always respond within 300ms.
16+
- Any conditions on any [Custom metric](/using-k6/metrics#custom-metrics).
1617

1718
Thresholds analyze the performance metrics and determine the final test result (pass/fail).
1819
Thresholds are a essential for [load-testing automation](/testing-guides/automated-performance-testing).
1920

20-
Here is a sample script that specifies two thresholds, one using a custom [Rate metric](/javascript-api/k6-metrics/rate), and one using a standard `http_req_duration` metric.
21+
Here is a sample script that specifies two thresholds, one evaluating the rate of http errors (`http_req_failed` metric) and one using the 95 percentile of all the response durations (the `http_req_duration` metric).
2122

2223
<CodeGroup labels={["threshold.js"]} lineNumbers={[true]}>
2324

2425
```javascript
2526
import http from 'k6/http';
26-
import { Rate } from 'k6/metrics';
27-
28-
const myFailRate = new Rate('failed requests');
2927

3028
export let options = {
3129
thresholds: {
32-
'failed requests': ['rate<0.1'], // threshold on a custom metric
33-
http_req_duration: ['p(95)<500'], // threshold on a standard metric
30+
http_req_failed: ['rate<0.01'], // http errors should be less than 1%
31+
http_req_duration: ['p(95)<500'], // 95% of requests should be below 200ms
3432
},
3533
};
3634

3735
export default function () {
38-
let res = http.get('https://test-api.k6.io/public/crocodiles/1/');
39-
myFailRate.add(res.status !== 200);
36+
http.get('https://test-api.k6.io/public/crocodiles/1/');
4037
}
4138
```
4239

4340
</CodeGroup>
4441

45-
The `failed requests` threshold specifies that we want our load test to be considered a failure
46-
(resulting in k6 exiting with a non-zero exit code) if 10% or more of requests resulted in the server
47-
returning anything else than a 200-response. The `http_req_duration` threshold specifies that
48-
95% of requests must complete within 500ms.
49-
5042
In other words, you specify the `pass` criteria when defining your threshold, and if that
5143
expression evaluates to `false` at the end of the test, the whole test will be considered a `fail`.
5244

5345
When executing that script, k6 will output something similar to this:
54-
![executing k6 with a threshold](images/Thresholds/executing-with-a-threshold.png)
5546

56-
- In the above case, criteria for both thresholds were met. The whole load test is considered
47+
<CodeGroup labels={["threshold-output"]} lineNumbers={[false]}>
48+
49+
```bash
50+
✓ http_req_duration..............: avg=151.06ms min=151.06ms med=151.06ms max=151.06ms p(90)=151.06ms p(95)=151.06ms
51+
{ expected_response:true }...: avg=151.06ms min=151.06ms med=151.06ms max=151.06ms p(90)=151.06ms p(95)=151.06ms
52+
✓ http_req_failed................: 0.00% ✓ 0 ✗ 1
53+
```
54+
</CodeGroup>
55+
56+
- In the above case, the criteria for both thresholds were met. The whole load test is considered
5757
to be a `pass`, which means that k6 will exit with exit code zero.
5858
59-
- If any of the thresholds had failed, the little green checkmark
60-
<span style="color:green; font-weight:bold">✓</span> next to the threshold name
61-
(`failed requests`, `http_req_duration`) would have been a red cross <span style="color:red; font-weight:bold">✗</span> instead, and k6 would have generated a non-zero exit code.
59+
- If any of the thresholds had failed, the little green checkmark <span style="color:green; font-weight:bold">✓</span> next to the threshold name
60+
(`http_req_failed`, `http_req_duration`) would have been a red cross <span style="color:red; font-weight:bold">✗</span> instead, and k6 would have generated a non-zero exit code.
6261
6362
## Copy-paste Threshold examples
6463
@@ -80,7 +79,29 @@ export let options = {
8079
};
8180

8281
export default function () {
83-
let res1 = http.get('https://test-api.k6.io/public/crocodiles/1/');
82+
http.get('https://test-api.k6.io/public/crocodiles/1/');
83+
sleep(1);
84+
}
85+
```
86+
87+
</CodeGroup>
88+
89+
<CodeGroup labels={["threshold-error-rate.js"]} lineNumbers={[true]}>
90+
91+
```javascript
92+
import http from 'k6/http';
93+
import { sleep } from 'k6';
94+
95+
export let options = {
96+
thresholds: {
97+
// During the whole test execution, the error rate must be lower than 1%.
98+
// `http_req_failed` metric is available since v0.31.0
99+
http_req_failed: ['rate<0.01'],
100+
},
101+
};
102+
103+
export default function () {
104+
http.get('https://test-api.k6.io/public/crocodiles/1/');
84105
sleep(1);
85106
}
86107
```

0 commit comments

Comments
 (0)