You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: framework/examples/myproject/README.md
+15-1Lines changed: 15 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,6 +14,8 @@ If the team decides on minimal or no manual testing and the project is trivial,
14
14
15
15
The team identifies potential integration points with third-party software, blockchain, and external services and document any testing limitations.
16
16
17
+
If the system is complex and the team wants to test parts of it using mocks, a schema should be provided to outline which components will be separated and include examples of the mocks.
18
+
17
19
If new components are required, the team implements them following this guide: [Developing Components](https://smartcontractkit.github.io/chainlink-testing-framework/developing/developing_components.html).
18
20
19
21
@@ -22,10 +24,14 @@ The team maintains a system-level smoke test where all components are deployed u
22
24
23
25
All on-chain changes are done through [chainlink-deployments](https://github.com/smartcontractkit/chainlink-deployments).
24
26
27
+
The test should not depend on `chainlink` core types.
28
+
25
29
The test is readable, and the README clearly explains what is tested.
26
30
27
31
The test is stable when run with a `-count 10`.
28
32
33
+
If the team has more than two distinct NodeSets (DONs), system-level tests in ephemeral environments should be limited to two NodeSets to allow for chaos testing later.
34
+
29
35
If your project includes multiple use cases and functionality suitable for end-to-end testing, you can add additional tests at this level.
30
36
31
37
## Level 2
@@ -38,7 +44,7 @@ This test deploys specific platform and plugin versions, performs an end-to-end
38
44
## Level 3
39
45
The team has a baseline performance testing suite.
40
46
41
-
At this level, the focus is not on improving performance but on detecting any performance degradation, supported by a reliable CI pipeline.
47
+
At this level, the focus is not on improving performance but on detecting any performance degradation early, supported by a reliable CI pipeline.
42
48
43
49
This pipeline runs as needed—nightly or before releases—enabling early detection of performance issues across all critical on-chain and off-chain functionality.
44
50
@@ -49,9 +55,17 @@ The team incorporates chaos engineering practices to test the system’s failure
49
55
50
56
This stage builds on [Level 3](#level-3), as it not only verifies that the system is reliable and can recover from reasonable failures but also ensures an understanding of how these failures impact performance and user experience.
51
57
58
+
Recommended test cases:
59
+
- Component failure (reboot)
60
+
- Slow network
61
+
- RPC node outage
62
+
- Stress test (limiting container resources)
63
+
52
64
## Level 5
53
65
The team has complete ownership of their persistent staging environment.
54
66
67
+
The team can use [components caching](https://smartcontractkit.github.io/chainlink-testing-framework/framework/components/caching.html) to reuse testing logic on staging and substitute staging environment URLs instead of an ephemeral environment.
68
+
55
69
They can perform upgrades, data migrations, and run advanced load tests to validate the scalability of their applications.
0 commit comments