Skip to content

Commit 3671c05

Browse files
MattDodsonEnglishsrperfppcano
authored
Feat: Rewrite test types explanations (#1136)
- More information - Longer, repeated structure - Updated images Co-authored-by: Señor Performo - Leandro Melendez <[email protected]> Co-authored-by: Pepe Cano <[email protected]>
1 parent 19a7291 commit 3671c05

33 files changed

+553
-452
lines changed

.vale/Vocab/docs/accept.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ APIs
4545
apm
4646
async
4747
backoff
48+
breakpoint
4849
browserified
4950
browserlist
5051
bundler

gatsby-node.js

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1585,6 +1585,12 @@ const createRedirects = ({ actions }) => {
15851585
isPermanent: true,
15861586
});
15871587

1588+
createRedirect({
1589+
fromPath: '/test-types/introduction',
1590+
toPath: '/test-types/load-test-types/',
1591+
isPermanent: true,
1592+
});
1593+
15881594
const redirects = {
15891595
'/javascript-api/k6-http/cookiejar-k6-http':
15901596
'/javascript-api/k6-http/cookiejar/',

src/data/markdown/translated-guides/en/06 Test Types/00 Introduction.md

Lines changed: 0 additions & 42 deletions
This file was deleted.
Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
---
2+
title: 'Load test types'
3+
excerpt: 'A series of conceptual articles explaining the different types of load tests. Learn about planning, running, and interpreting different tests for different performance goals.'
4+
---
5+
6+
Many things can go wrong when a system is under load.
7+
The system must run numerous operations simultaneously and respond to different requests from a variable number of users.
8+
To prepare for these performance risks, teams use load testing.
9+
10+
But a good load-testing strategy requires more than just executing a single script.
11+
Different patterns of traffic create different risk profiles for the application.
12+
For comprehensive preparation, teams must test the system against different _test types_.
13+
14+
![Overview of load test shapes](./images/chart-load-test-types-overview.png)
15+
16+
## Different tests for different goals
17+
18+
Start with smoke tests, then progress to higher loads and longer durations.
19+
20+
The main types are as follows. Each type has its own article outlining its essential concepts.
21+
22+
- [**Smoke tests**](../smoke-testing) validate that your script works and that the system performs adequately under minimal load.
23+
24+
- [**Average-load test**](../load-testing) assess how your system performs under expected normal conditions.
25+
26+
- [**Stress tests**](../stress-testing) assess how a system performs at its limits when load exceeds the expected average.
27+
28+
- [**Soak tests**](../soak-testing) assess the reliability and performance of your system over extended periods.
29+
30+
- [**Spike tests**](../spike-testing) validate the behavior and survival of your system in cases of sudden, short, and massive increases in activity.
31+
32+
- [**Breakpoint tests**](../breakpoint-testing) gradually increase load to identify the capacity limits of the system.
33+
34+
<Blockquote mod="note" title="">
35+
36+
In k6 scripts, configure your load in the [`options` object](/using-k6/k6-options/reference/) or using [Scenarios](/using-k6/scenarios). The workload configuration is separated from the logic of VUs.
37+
38+
</Blockquote>
39+
40+
## Test-type cheat sheet
41+
42+
The following table provides some broad comparisons.
43+
44+
| Type | VUs/Throughput | Duration | When? |
45+
|------------|-----------------------|----------------------------|------------------------------------------------------------------------------------------------------------------|
46+
| Smoke | Low | Quick (seconds or minutes) | Every time new code or a change is done. It checks scripts, baseline system metrics, and deviations from changes |
47+
| Load | Average production | Mid (15-60 minutes) | Often to check system maintains performance with average use |
48+
| Stress | High (above average) | Mid (15-60 minutes) | When system may receive above-average loads to check how it manages |
49+
| Soak | Average | Long (hours) | After changes to check system under prolonged continuous use |
50+
| Spike | Very high | Quick (seconds to minutes) | Rarely, when system risks sudden rush |
51+
| Breakpoint | Increases until break | As long as necessary | A few times to find the upper limits of the system |
52+
53+
54+
## General recommendations
55+
56+
When you write and run different test types in k6, consider the following.
57+
58+
### Start with a smoke test
59+
60+
Start with a [smoke test](../smoke-testing).
61+
Before beginning larger tests, validate that your scripts work as expected and that your system performs well with a few users.
62+
63+
After you know that the script works and the system responds correctly to minimal load,
64+
you can move on to average-load tests.
65+
From there, you can progress to more complex load patterns.
66+
67+
### The specifics depend on your use case
68+
69+
Systems have different architectures and different user bases. As a result, the correct load testing strategy is highly dependent on the risk profile for your organization. Avoid thinking in absolutes.
70+
71+
For example, k6 can model load by either number of VUs or by number of iterations per second ([open vs. closed](https://k6.io/docs/using-k6/scenarios/concepts/open-vs-closed/)).
72+
When you design your test, consider which pattern makes sense for the type.
73+
74+
What's more, **no single test type eliminates all risk.**
75+
To assess different failure modes of your system, incorporate multiple test types.
76+
The risk profile of your system determines what test types to emphasize:
77+
- Some systems are more at risk of longer use, in which case soaks should be prioritized.
78+
- Others are more at risk of intensive use, in which case stress tests should take precedence.
79+
80+
In any case, **no single test can uncover all issues**.
81+
82+
What's more, the categories themselves are relative to use cases. A stress test for one application is an average-load test for another. Indeed, no consensus even exists about the names of these test types (each of the following topics provides alternative names).
83+
84+
### Aim for simple designs and reproducible results
85+
86+
While the specifics are greatly context-dependent, what's constant is that you want to make results that you can compare and interpret.
87+
88+
Stick to simple load patterns. For all test types, directions is enough: ramp-up, plateau, ramp-down.
89+
90+
Avoid "rollercoaster" series where load increases and decreases multiple times. These will waste resources and make it hard to isolate issues.
91+

src/data/markdown/translated-guides/en/06 Test Types/01 Smoke Testing.md

Lines changed: 54 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -1,74 +1,81 @@
11
---
22
title: "Smoke testing"
33
head_title: 'What is Smoke Testing? How to create a Smoke Test in k6'
4-
excerpt: "Smoke test is a regular load test, configured for minimal load. You want to run a smoke test as a sanity check every time you write a new
5-
script or modify an existing script. Let’s see an example."
4+
excerpt: "A Smoke test is a minimal load test to run when you create or modify a script."
65
---
76

8-
A smoke test is a test configured for minimal load.
7+
Smoke tests have a minimal load.
8+
Run them to verify that the script works well, the system functions under minimal load, and to gather baseline performance values.
99

10-
Each time you write a new test script, it's a good idea to run a smoke to test first, which helps you:
11-
- Verify that your test script doesn't have errors.
12-
- Verify that your system doesn't throw any errors when under minimal load.
10+
This test type consists of running tests with only one or a few VUs. Keep the number to 5 VUs or less.
11+
Exceeding 5 could turn the test into a mini-load test.
12+
Similarly, the test shouldn't have many iterations. 3 to 10 iterations should be enough (or an iteration duration with a short period).
1313

14-
## Smoke testing in k6
14+
![Overview of a smoke test](images/chart-smoke-test-overview.png)
1515

16-
The following script is a sample smoke test script to get you started.
17-
You can copy it, change the end points, and start testing.
18-
If you'd like to see more comprehensive script, check out our [example section](/examples).
16+
In some testing conversation, smoke tests are also called shakeout tests.
1917

20-
<CodeGroup labels={["sample-smoke-test.js"]} lineNumbers={[true]} heightTogglers={[true]}>
18+
## When to run a Smoke test
2119

22-
```javascript
23-
import http from 'k6/http';
24-
import { check, group, sleep, fail } from 'k6';
20+
Teams should run smoke tests whenever a test script is created or updated. Smoke testing should also be done every time the application code is updated.
2521

26-
export const options = {
27-
vus: 1, // 1 user looping for 1 minute
28-
duration: '1m',
22+
It's a good practice to run a smoke test as a first step, with the following goals:
2923

30-
thresholds: {
31-
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
32-
},
33-
};
24+
- Verify that your test script doesn't have errors.
25+
- Verify that your system doesn't throw any errors (performance or system related) when under minimal load.
26+
- Gather baseline performance metrics of your system’s response under minimal load.
27+
- With simple logic, to serve as a synthetic test to monitor the performance and availability of production environments.
3428

35-
const BASE_URL = 'https://test-api.k6.io';
36-
const USERNAME = 'TestUser';
37-
const PASSWORD = 'SuperCroc2020';
29+
## Considerations
30+
31+
When you prepare a smoke test, consider the following:
3832

39-
export default () => {
40-
const loginRes = http.post(`${BASE_URL}/auth/token/login/`, {
41-
username: USERNAME,
42-
password: PASSWORD,
43-
});
4433

45-
check(loginRes, {
46-
'logged in successfully': (resp) => resp.json('access') !== '',
47-
});
34+
- **Each time you create or update a script, run a smoke test**
4835

49-
const authHeaders = {
50-
headers: {
51-
Authorization: `Bearer ${loginRes.json('access')}`,
52-
},
53-
};
36+
Because smoke tests verify test scripts, try to run one every time you create or update a script. Avoid running other test types with untested scripts.
37+
38+
- **Keep throughput small and duration short**
39+
40+
Configure your test script to be executed by a small number of VUs (from 2 to 5) with few iterations (3 to 10) or brief durations (30 to 60 seconds).
41+
42+
## Smoke testing in k6
5443

55-
const myObjects = http.get(`${BASE_URL}/my/crocodiles/`, authHeaders).json();
56-
check(myObjects, { 'retrieved crocodiles': (obj) => obj.length > 0 });
44+
<CodeGroup labels={["smoke.js"]} lineNumbers={[]} showCopyButton={[true]}>
5745

46+
```javascript
47+
import http from 'k6/http';
48+
import { check, sleep} from 'k6';
49+
50+
export const options = {
51+
vus: 3, // Key for Smoke test. Keep it at 2, 3, max 5 VUs
52+
duration: '1m', // This can be shorter or just a few iterations
53+
};
54+
55+
export default () => {
56+
const urlRes = http.req('https://test-api.k6.io');
5857
sleep(1);
58+
// MORE STEPS
59+
// Here you can have more steps or complex script
60+
// Step1
61+
// Step2
62+
// etc.
5963
};
6064
```
6165

6266
</CodeGroup>
6367

64-
The VU chart of a smoke test should look similar to this.
65-
You want to use only 1 or 2 VUs.
66-
![Smoke test VU chart](./images/smoke-test.png)
6768

68-
If your smoke test produces any errors,
69-
you should either correct the script or fix your environment before you continue.
69+
The following script is an example smoke test. You can copy it, change the endpoints, and start testing. For more comprehensive test logic, refer to [Examples](/examples).
70+
The VU chart of a smoke test should look similar to this. Again, a smoke test should use only 2 or 3 VUs and run for only a brief period.
71+
72+
![The shape of the smoke test as configured in the preceding script](images/chart-smoke-test-k6-script-example.png)
73+
74+
## Results analysis
75+
76+
The smoke test initially validates that your script runs without errors. If any script-related errors appear, correct the script before trying any more extensive tests.
77+
78+
On the other hand, if you notice poor performance with these low VU numbers, report it, fix your environment, and try again with a smoke test before any further tests.
7079

71-
The k6 output should look similar to this:
72-
![Smoke test Terminal Output](./images/smoke-test-terminal-output.png)
80+
Once your smoke test shows zero errors and the performance results seem acceptable, you can proceed to other test types.
7381

74-
Once your smoke test shows zero errors, you can go to the next step and execute a [load test](/test-types/load-testing) to assess the performance of your system.

0 commit comments

Comments
 (0)