Skip to content

Commit 4877835

Browse files
skudasovskudasov
authored
Upgrade tests (#1311)
upgrade tests
1 parent 81ed16b commit 4877835

File tree

12 files changed

+259
-54
lines changed

12 files changed

+259
-54
lines changed

‎.github/workflows/docs.yml‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,6 @@ name: GH Pages Deploy (Docs, mdbook)
33

44
on:
55
push:
6-
branches:
7-
- main
86
tags:
97
- '*'
108

‎.github/workflows/framework-golden-tests.yml‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,12 @@ jobs:
3030
config: chaos.toml
3131
count: 1
3232
timeout: 10m
33-
- name: TestReload
34-
config: reload.toml
33+
- name: TestUpgradeAll
34+
config: upgrade_all.toml
35+
count: 1
36+
timeout: 10m
37+
- name: TestUpgradeSome
38+
config: upgrade_some.toml
3539
count: 1
3640
timeout: 10m
3741
steps:

‎book/src/SUMMARY.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
- [Test Configuration](./framework/test_configuration_overrides.md)
1717
- [Components Persistence](framework/components/state.md)
1818
- [Components Caching](framework/components/caching.md)
19+
- [External Environment](framework/components/external.md)
1920
- [Secrets]()
2021
- [Docker](framework/docker.md)
2122
- [Observability Stack](framework/observability/observability_stack.md)
@@ -38,7 +39,7 @@
3839
- [Chainlink]()
3940
- [RPC]()
4041
- [Loki]()
41-
- [Testing]()
42+
- [Testing](framework/testing.md)
4243
- [Smoke]()
4344
- [Performance]()
4445
- [Chaos]()

‎book/src/framework/components/caching.md‎

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -9,21 +9,3 @@ If cached config has any outputs with `use_cache = true` it will be used instead
99
```
1010
export CTF_CONFIGS=smoke-cache.toml
1111
```
12-
13-
### Using remote components
14-
15-
Because components are decoupled through outputs, you can use a cached config and switch outputs to any deployed infrastructure, such as staging. This allows you to reuse the same testing logic for behavior validation.
16-
17-
Example:
18-
```
19-
[blockchain_a.out]
20-
use_cache = true
21-
chain_id = '31337'
22-
23-
[[blockchain_a.out.nodes]]
24-
ws_url = 'ws://127.0.0.1:33447'
25-
http_url = 'http://127.0.0.1:33447'
26-
docker_internal_ws_url = 'ws://anvil-3716a:8900'
27-
docker_internal_http_url = 'http://anvil-3716a:8900'
28-
```
29-
Set flag `use_cache = true` on any component output, change output fields as needed and run your test again.

‎book/src/framework/components/external.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# External Environment
2+
3+
### Using remote components
4+
5+
Because components are decoupled through outputs, you can use a cached config and switch outputs to any deployed infrastructure, such as staging.
6+
7+
This allows you to reuse the same testing logic for behavior validation.
8+
9+
For example, to integrate with remote `k8s` environment you can use `CTF_CONFIGS=smoke_external.toml` and override all the outputs of components to connect to your remote env.
10+
11+
```toml
12+
[blockchain_a]
13+
14+
[blockchain_a.out]
15+
chain_id = "31337"
16+
use_cache = true
17+
18+
[[blockchain_a.out.nodes]]
19+
# set up your RPC URLs
20+
docker_internal_http_url = "http://anvil-2cfcf:8545"
21+
docker_internal_ws_url = "ws://anvil-2cfcf:8545"
22+
http_url = "http://127.0.0.1:8545"
23+
ws_url = "ws://127.0.0.1:8545"
24+
25+
[contracts]
26+
27+
[contracts.out]
28+
# set up your contracts
29+
addresses = ["0x5fbdb2315678afecb367f032d93f642f64180aa3"]
30+
use_cache = true
31+
32+
[data_provider]
33+
port = 9111
34+
35+
[data_provider.out]
36+
# setup your data provider URLs
37+
base_url_docker = "http://host.docker.internal:9111"
38+
base_url_host = "http://localhost:9111"
39+
40+
[nodeset]
41+
42+
[[nodeset.node_specs]]
43+
44+
[nodeset.out]
45+
use_cache = true
46+
47+
[[nodeset.out.cl_nodes]]
48+
use_cache = true
49+
50+
[nodeset.out.cl_nodes.node]
51+
# set up each node URLs
52+
docker_internal_url = "http://node0:6688"
53+
p2p_docker_internal_url = "http://node0:6690"
54+
p2p_url = "http://127.0.0.1:12000"
55+
url = "http://127.0.0.1:10000"
56+
57+
[nodeset.out.cl_nodes.postgresql]
58+
# set up a database URL so tests can connect to your database if needed
59+
url = "postgresql://chainlink:[email protected]:13000/db_0?sslmode=disable"
60+
61+
# more nodes in this array, configuration is the same ...
62+
```

‎book/src/framework/testing.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# "Golden" Templates
2+
3+
[Here](https://github.com/smartcontractkit/chainlink-testing-framework/actions/runs/11739154666/job/32703095118?pr=1311) are our "golden" templates for end-to-end tests, covering every test type:
4+
5+
- `Smoke`
6+
- `Performance`
7+
- `Chaos`
8+
- `Upgrade`
9+
10+
These tests act as a maturity model and are implemented across all our products.
11+
12+
Use this [workflow](https://github.com/smartcontractkit/chainlink-testing-framework/actions/runs/11739154666/workflow?pr=1311) as a starting point for developing a new end-to-end integration test.
13+
14+
Set the count to 5-10 during development, and once stable, set the timeout and proceed to merge.

‎framework/.changeset/v0.2.0.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
- Add self-test "golden" set of end-to-end tests https://github.com/smartcontractkit/chainlink-testing-framework/actions/runs/11739154666/workflow?pr=1311
2+
- Add upgrade tests
3+
- Wrapped reboot helpers for NodeSet

‎framework/components/simple_node_set/reload.go‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,20 @@
11
package simple_node_set
22

3-
// UpdateNodeConfigs updates nodes configuration TOML files
3+
import (
4+
"github.com/smartcontractkit/chainlink-testing-framework/framework/chaos"
5+
"github.com/smartcontractkit/chainlink-testing-framework/framework/components/blockchain"
6+
"time"
7+
)
8+
9+
// UpgradeNodeSet updates nodes configuration TOML files
410
// this API is discouraged, however, you can use it if nodes require restart or configuration updates, temporarily!
5-
func UpdateNodeConfigs(in *Input, cfg string) {
6-
in.NodeSpecs[0].Node.UserConfigOverrides = in.NodeSpecs[0].Node.UserConfigOverrides + cfg
11+
func UpgradeNodeSet(in *Input, bc *blockchain.Output, url string, wait time.Duration) (*Output, error) {
12+
_, err := chaos.ExecPumba("rm --volumes=false re2:node.*|postgresql.*", wait)
13+
if err != nil {
14+
return nil, err
15+
}
716
in.Out = nil
17+
out, err := NewSharedDBNodeSet(in, bc, url)
18+
in.Out = out
19+
return out, err
820
}

framework/examples/myproject/reload.toml renamed to framework/examples/myproject/upgrade_all.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,5 +19,5 @@
1919
pull_image = true
2020

2121
[nodeset.node_specs.node]
22-
image = "public.ecr.aws/chainlink/chainlink:v2.17.0"
23-
pull_image = false
22+
image = "public.ecr.aws/chainlink/chainlink:v2.16.0"
23+
pull_image = false

‎framework/examples/myproject/upgrade_all_test.go‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
package examples
2+
3+
import (
4+
"fmt"
5+
"github.com/smartcontractkit/chainlink-testing-framework/framework"
6+
"github.com/smartcontractkit/chainlink-testing-framework/framework/clclient"
7+
"github.com/smartcontractkit/chainlink-testing-framework/framework/components/blockchain"
8+
"github.com/smartcontractkit/chainlink-testing-framework/framework/components/fake"
9+
ns "github.com/smartcontractkit/chainlink-testing-framework/framework/components/simple_node_set"
10+
"github.com/stretchr/testify/require"
11+
"testing"
12+
"time"
13+
)
14+
15+
type CfgUpgradeAll struct {
16+
BlockchainA *blockchain.Input `toml:"blockchain_a" validate:"required"`
17+
MockerDataProvider *fake.Input `toml:"data_provider" validate:"required"`
18+
NodeSet *ns.Input `toml:"nodeset" validate:"required"`
19+
}
20+
21+
func TestUpgradeAll(t *testing.T) {
22+
in, err := framework.Load[CfgUpgradeAll](t)
23+
require.NoError(t, err)
24+
25+
bc, err := blockchain.NewBlockchainNetwork(in.BlockchainA)
26+
require.NoError(t, err)
27+
dp, err := fake.NewFakeDataProvider(in.MockerDataProvider)
28+
require.NoError(t, err)
29+
30+
// deploy first time
31+
out, err := ns.NewSharedDBNodeSet(in.NodeSet, bc, dp.BaseURLDocker)
32+
require.NoError(t, err)
33+
34+
c, err := clclient.NewCLDefaultClients(out.CLNodes, framework.L)
35+
require.NoError(t, err)
36+
_, _, err = c[0].CreateJobRaw(testJob)
37+
require.NoError(t, err)
38+
39+
in.NodeSet.NodeSpecs[0].Node.Image = "public.ecr.aws/chainlink/chainlink:v2.17.0"
40+
in.NodeSet.NodeSpecs[0].Node.UserConfigOverrides = `
41+
[Log]
42+
level = 'info'
43+
`
44+
45+
out, err = ns.UpgradeNodeSet(in.NodeSet, bc, dp.BaseURLDocker, 5*time.Second)
46+
require.NoError(t, err)
47+
48+
jobs, _, err := c[0].ReadJobs()
49+
require.NoError(t, err)
50+
fmt.Println(jobs)
51+
52+
t.Run("test something", func(t *testing.T) {
53+
for _, n := range out.CLNodes {
54+
require.NotEmpty(t, n.Node.HostURL)
55+
require.NotEmpty(t, n.Node.HostP2PURL)
56+
}
57+
})
58+
}

0 commit comments

Comments
 (0)