You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Here](https://github.com/smartcontractkit/chainlink-testing-framework/actions/runs/11739154666/job/32703095118?pr=1311) are our "golden" templates for end-to-end tests, covering every test type:
3
+
Read this [doc](https://github.com/smartcontractkit/chainlink-testing-framework/tree/main/framework/examples/myproject) to understand the rationale behind our testing approach and to explore the stages of maturity in end-to-end testing.
4
4
5
-
-`Smoke`
6
-
-`PerformanceBaseline`
7
-
-`Chaos`
8
-
-`Upgrade`
9
-
10
-
These tests act as a maturity model and are implemented across all our products.
11
-
12
-
Refer to this README to understand the rationale behind our testing approach and to explore the stages of maturity in end-to-end testing.
13
-
14
-
## Developing
15
-
Run the tests locally
16
-
```
17
-
CTF_CONFIGS=smoke.toml go test -v -run TestSmoke
18
-
CTF_CONFIGS=performance_baseline.toml go test -v -run TestPerformanceBaseline
19
-
CTF_CONFIGS=chaos.toml go test -v -run TestChaos
20
-
CTF_CONFIGS=upgrade.toml go test -v -run TestUpgrade
21
-
```
5
+
The following chapters detail specific testing types and best practices for system-level end-to-end testing.
The team maintains a system-level smoke test where all components are deployed using `docker`.
11
+
12
+
All on-chain changes are done through [chainlink-deployments](https://github.com/smartcontractkit/chainlink-deployments).
13
+
14
+
The test is readable, and the README clearly explains its purpose.
15
+
16
+
The test is reliable and stable when run with a `-count 30`.
17
+
18
+
If your project includes multiple use cases and functionality suitable for end-to-end testing, you can add additional tests at this level.
19
+
20
+
## Level 2
21
+
The team has an "upgrade" test to verify product compatibility with older versions.
22
+
23
+
While the number of compatible versions is team-determined, identifying incompatibilities at the system level early is a valuable, mature practice.
24
+
25
+
This test deploys specific platform and plugin versions, performs an end-to-end smoke test, and then upgrades (or migrates) the plugin(s) or platform on the same database to ensure that users remain unaffected by the upgrade.
26
+
27
+
## Level 3
28
+
The team has a baseline performance testing suite.
29
+
30
+
At this level, the focus is not on improving performance but on detecting any performance degradation, supported by a reliable CI pipeline.
31
+
32
+
This pipeline runs as needed—nightly or before releases—enabling early detection of performance issues across all critical on-chain and off-chain functionality.
33
+
34
+
This stage combines performance testing with observability enhancements. The team should have fundamental system-level performance tests in place, along with dashboards to monitor product-specific metrics.
35
+
36
+
## Level 4
37
+
The team incorporates chaos engineering practices to test the system’s failure modes.
38
+
39
+
This stage builds on [Level 3](#level-3), as it not only verifies that the system is reliable and can recover from reasonable failures but also ensures an understanding of how these failures impact performance and user experience.
40
+
41
+
## Level 5
42
+
The team has complete ownership of their persistent staging environment.
43
+
44
+
They can perform upgrades, data migrations, and run advanced load tests to validate the scalability of their applications.
45
+
3
46
## Developing
4
47
Run the tests locally
5
48
```
6
49
CTF_CONFIGS=smoke.toml go test -v -run TestSmoke
50
+
CTF_CONFIGS=upgrade.toml go test -v -run TestUpgrade
7
51
CTF_CONFIGS=performance_baseline.toml go test -v -run TestPerformanceBaseline
8
52
CTF_CONFIGS=chaos.toml go test -v -run TestChaos
9
-
CTF_CONFIGS=upgrade.toml go test -v -run TestUpgrade
53
+
CTF_CONFIGS=scalability.toml go test -v -run TestScalability
0 commit comments