Skip to content

Commit bc5f730

Browse files
skudasovskudasov
authored
stateful db for chaos tests (#1303)
* stateful docker containers, static port ranges, reload config example
1 parent 3c5e76a commit bc5f730

File tree

29 files changed

+417
-92
lines changed

29 files changed

+417
-92
lines changed

‎.github/workflows/framework-golden-tests.yml‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,6 @@
11
name: Framework Golden Tests Examples
22
on:
33
push:
4-
concurrency:
5-
group: ${{ github.workflow }}-${{ github.ref }}-framework-golden-examples
6-
cancel-in-progress: true
74

85
jobs:
96
test:
@@ -18,6 +15,7 @@ jobs:
1815
id-token: write
1916
contents: read
2017
strategy:
18+
fail-fast: false
2119
matrix:
2220
test:
2321
- name: TestSmoke
@@ -32,6 +30,10 @@ jobs:
3230
config: chaos.toml
3331
count: 1
3432
timeout: 10m
33+
- name: TestReload
34+
config: reload.toml
35+
count: 1
36+
timeout: 10m
3537
steps:
3638
- name: Checkout repo
3739
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
@@ -62,9 +64,10 @@ jobs:
6264
path: |
6365
~/.cache/go-build
6466
~/go/pkg/mod
65-
key: go-modules-${{ hashFiles('**/go.sum') }}-${{ runner.os }}-framework-golden-examples
67+
key: go-modules-${{ hashFiles('framework/examples/myproject/go.sum') }}-${{ runner.os }}-framework-golden-examples
6668
restore-keys: |
67-
go-modules-${{ hashFiles('**/go.sum') }}-${{ runner.os }}-framework-golden-examples
69+
go-modules-${{ runner.os }}-framework-golden-examples
70+
go-modules-${{ runner.os }}
6871
- name: Install dependencies
6972
run: go mod download
7073
- name: Run Docker Component Tests

‎book/src/SUMMARY.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,10 @@
1414
- [CLI](./framework/cli.md)
1515
- [Configuration](./framework/configuration.md)
1616
- [Test Configuration](./framework/test_configuration_overrides.md)
17-
- [Caching](framework/components/caching.md)
17+
- [Components Persistence](framework/components/state.md)
18+
- [Components Caching](framework/components/caching.md)
1819
- [Secrets]()
20+
- [Docker](framework/docker.md)
1921
- [Observability Stack](framework/observability/observability_stack.md)
2022
- [Metrics](framework/observability/metrics.md)
2123
- [Logs](framework/observability/logs.md)

‎book/src/ci/ci.md‎

Lines changed: 54 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -2,31 +2,65 @@
22

33
Here we describe our good practices for structuring different types of tests in Continuous Integration (GitHub Actions).
44

5-
The simplest flow can look like:
6-
75
Set up secrets in your GitHub repository
86
```
97
gh secret set CTF_SIMULATED_KEY_1 --body "..."
108
```
119

1210
Add a workflow
1311
```yaml
14-
name: Run E2E tests
15-
12+
name: Framework Golden Tests Examples
1613
on:
1714
push:
1815

1916
jobs:
2017
test:
21-
runs-on: ubuntu-latest
18+
defaults:
19+
run:
20+
working-directory: framework/examples/myproject
2221
env:
23-
CTF_CONFIGS: smoke.toml
24-
CTF_LOG_LEVEL: info
25-
CTF_LOKI_STREAM: "false"
26-
PRIVATE_KEY: ${{ secrets.CTF_SIMULATED_KEY_1 }}
22+
LOKI_TENANT_ID: promtail
23+
LOKI_URL: http://localhost:3030/loki/api/v1/push
24+
runs-on: ubuntu-latest
25+
permissions:
26+
id-token: write
27+
contents: read
28+
strategy:
29+
fail-fast: false
30+
matrix:
31+
test:
32+
- name: TestSmoke
33+
config: smoke.toml
34+
count: 1
35+
timeout: 10m
36+
- name: TestLoad
37+
config: load.toml
38+
count: 1
39+
timeout: 10m
40+
- name: TestChaos
41+
config: chaos.toml
42+
count: 1
43+
timeout: 10m
2744
steps:
28-
- name: Check out code
29-
uses: actions/checkout@v3
45+
- name: Checkout repo
46+
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
47+
- name: Configure AWS credentials using OIDC
48+
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
49+
with:
50+
role-to-assume: ${{ secrets.PUBLIC_AWS_ECR_ROLE }}
51+
aws-region: us-east-1
52+
- name: Authenticate to ECR Public
53+
id: login-ecr-public
54+
uses: aws-actions/amazon-ecr-login@062b18b96a7aff071d4dc91bc00c4c1a7945b076 # v2.0.1
55+
with:
56+
registry-type: public
57+
- name: Check for changes in Framework
58+
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
59+
id: changes
60+
with:
61+
filters: |
62+
src:
63+
- 'framework/**'
3064
- name: Set up Go
3165
uses: actions/setup-go@v4
3266
with:
@@ -37,14 +71,18 @@ jobs:
3771
path: |
3872
~/.cache/go-build
3973
~/go/pkg/mod
40-
key: go-modules-${{ hashFiles('**/go.sum') }}-${{ runner.os }}
74+
key: go-modules-${{ hashFiles('framework/examples/myproject/go.sum') }}-${{ runner.os }}-framework-golden-examples
4175
restore-keys: |
42-
go-modules-${{ hashFiles('**/go.sum') }}-${{ runner.os }}
76+
go-modules-${{ runner.os }}-framework-golden-examples
77+
go-modules-${{ runner.os }}
4378
- name: Install dependencies
4479
run: go mod download
45-
- name: Run tests
46-
working-directory: e2e/capabilities
47-
run: go test -v -run TestDON
80+
- name: Run Docker Component Tests
81+
if: steps.changes.outputs.src == 'true'
82+
env:
83+
CTF_CONFIGS: ${{ matrix.test.config }}
84+
run: |
85+
go test -timeout ${{ matrix.test.timeout }} -v -count ${{ matrix.test.count }} -run ${{ matrix.test.name }}
4886
```
4987
5088
If you need to structure a lot of different end-to-end tests follow [this](https://github.com/smartcontractkit/.github/tree/main/.github/workflows) guide.

‎book/src/framework/components/caching.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
## Component caching
22

3-
We use component caching to accelerate test development and enforce idempotent test actions.
3+
We use component caching to accelerate test development and enforce idempotent test actions development.
44

55
Each component is isolated by means of inputs and outputs.
66

‎book/src/framework/components/chainlink/nodeset.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,11 @@ Then configure NodeSet
4848
# defines how we override configs, either we apply first node fields to all of them
4949
# or we define each node custom configuration (used in compatibility testing)
5050
override_mode = "all"
51+
# HTTP API port range start, each new node get port incremented (host machine)
52+
http_port_range_start = 10000
53+
# P2P API port range start, each new node get port incremented (host machine)
54+
p2p_port_range_start = 12000
55+
5156

5257
[[nodeset.node_specs]]
5358
# Optional URL for fake data provider URL
@@ -59,6 +64,8 @@ Then configure NodeSet
5964
image = "postgres:15.6"
6065
# Pulls the image every time if set to 'true', used like that in CI. Can be set to 'false' to speed up local runs
6166
pull_image = true
67+
# PostgreSQL volume name
68+
volume_name = ""
6269

6370
[nodeset.node_specs.node]
6471
# A list of paths to capability binaries

‎book/src/framework/components/databases/pg.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# PostgreSQL

‎book/src/framework/components/state.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Components Persistence
2+
3+
We use static port ranges and volumes for all components to simplify Docker port management for developers.
4+
5+
This approach allows us to apply chaos testing to any container, ensuring it reconnects and retains the data needed for your tests.
6+
7+
When deploying a component, you can explicitly configure port ranges if the default ports don’t meet your needs.
8+
9+
Defaults are:
10+
- [NodeSet](../components/chainlink/nodeset.md) (Node HTTP API): `10000..100XX`
11+
- [NodeSet](../components/chainlink/nodeset.md) (Node P2P API): `12000..120XX`
12+
```
13+
[nodeset]
14+
# HTTP API port range start, each new node get port incremented (host machine)
15+
http_port_range_start = 10000
16+
# P2P API port range start, each new node get port incremented (host machine)
17+
p2p_port_range_start = 12000
18+
```
19+
- [PostgreSQL](../components/chainlink/nodeset.md): `13000` (we do not allow to have multiple databases for now, for simplicity)
20+
```
21+
[nodeset.node_specs.db]
22+
# PostgreSQL volume name
23+
volume_name = "a"
24+
# PostgreSQL port (host machine)
25+
port = 13000
26+
```
27+
28+
When you run `ctf d rm` database volume will be **removed**.
29+
30+
<div class="warning">
31+
32+
One node set is enough for any kind of testing, if you need more nodes consider extending your existing node set:
33+
```
34+
[nodeset]
35+
nodes = 10
36+
```
37+
</div>

‎book/src/framework/docker.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Docker
2+
3+
We are not removing volumes and images when you are working locally to allow you to debug, however, to clean up some space use:
4+
```
5+
docker volume prune -f
6+
docker system prune -f
7+
```

‎book/src/framework/first_test.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,6 @@ Summary:
5555
- We defined configuration for `BlockchainNetwork`
5656
- We've used one CTF component in test and checked if it's working
5757

58-
You can learn more about [component design](./components/overview.md) or proceed with another example of [connecting Chainlink node](./connecting_chainlink_node.md)
58+
Now let's connect the [Chainlink](./connecting_chainlink_node.md) node!
5959

6060
Learn more about [anvil](./components/blockchains/anvil.md) component.

‎book/src/framework/nodeset_environment.md‎

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -72,13 +72,9 @@ Run it
7272
CTF_CONFIGS=smoke.toml go test -v -run TestNodeSet
7373
```
7474

75-
You'll see something like, use any URL to access CL node
75+
Check the logs to access the UI
7676
```bash
77-
6:14PM INF Chainlink node url URL=http://127.0.0.1:34041
78-
6:14PM INF Chainlink node url URL=http://127.0.0.1:34045
79-
6:14PM INF Chainlink node url URL=http://127.0.0.1:34044
80-
6:14PM INF Chainlink node url URL=http://127.0.0.1:34042
81-
6:14PM INF Chainlink node url URL=http://127.0.0.1:34043
77+
12:41AM INF UI=["http://127.0.0.1:10000","http://127.0.0.1:10001", ...]
8278
```
8379

8480
Use credentials to authorize:

0 commit comments

Comments
 (0)