Merge branch 'main' into TT-1830

Author: lukaszcl

.github/workflows/framework-golden-tests.yml

@@ -22,22 +22,23 @@ jobs:
             config: smoke.toml
             count: 1
             timeout: 10m
-          - name: TestLoad
-            config: load.toml
+          - name: TestUpgrade
+            config: upgrade.toml
             count: 1
             timeout: 10m
-          - name: TestChaos
-            config: chaos.toml
+          - name: TestPerformanceBaseline
+            config: performance_baseline.toml
             count: 1
             timeout: 10m
-          - name: TestUpgradeAll
-            config: upgrade_all.toml
+          - name: TestChaos
+            config: chaos.toml
             count: 1
             timeout: 10m
-          - name: TestUpgradeSome
-            config: upgrade_some.toml
+          - name: TestScalability
+            config: scalability.toml
             count: 1
             timeout: 10m
+    name: ${{ matrix.test.name }}
     steps:
       - name: Checkout repo
         uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4

.github/workflows/modgraph.yaml

@@ -0,0 +1,15 @@
+name: Module graph
+on:
+  pull_request:
+jobs:
+  modgraph:
+    runs-on: ubuntu-latest
+    name: modgraph
+    steps:
+      - name: Checkout the Repo
+        uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # v4.2.1
+      - name: Setup Go
+        uses: actions/setup-go@v5
+      - run: make modgraph
+      - name: Ensure clean modgraph
+        run: git diff --minimal --exit-code

.gitignore

@@ -46,9 +46,7 @@ suite/remote_runner_config.yaml
 *.csv
 
 # docs
-docs/_site/
-docs/.sass-cache/
-docs/Gemfile.lock
+book/docs/
 
 dist/
 
@@ -78,4 +76,4 @@ __debug*
 .tool-versions
 
 import_keys_test.go
-tag.py
+tag.py

.nancy-ignore

@@ -11,3 +11,5 @@ CVE-2023-48795 # Improper Validation of Integrity Check Value x/crypto, which is
 CVE-2024-24786 # CWE-835 Loop with Unreachable Exit Condition ('Infinite Loop') in indirect dependency
 CVE-2024-32972 # CWE-400: Uncontrolled Resource Consumption ('Resource Exhaustion') [still not fixed, not even in v1.13.8]
 CVE-2023-42319 # CWE-noinfo: lol... go-ethereum v1.13.8 again
+CVE-2024-10086 # Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting')
+CVE-2024-51744 # CWE-755: Improper Handling of Exceptional Conditions

CONTRIBUTING.md

@@ -0,0 +1,5 @@
+.PHONY: modgraph
+modgraph:
+	go install github.com/jmank88/[email protected]
+	go install github.com/jmank88/[email protected]
+	./modgraph > go.md

Makefile

@@ -1,15 +1,18 @@
-## MDBook
-Install [mdbook](https://github.com/rust-lang/mdBook)
-```
+# Chainlink Testing Framework Docs
+
+We use [mdBook](https://github.com/rust-lang/mdBook) for our docs. They can be found [here](https://smartcontractkit.github.io/chainlink-testing-framework/).
+
+## Development
+
+First [install Rust](https://doc.rust-lang.org/cargo/getting-started/installation.html), then [mdbook](https://github.com/rust-lang/mdBook).
+
+```sh
+# Install mdBook
 cargo install mdbook && \
 cargo install mdbook-alerts && \
 cargo install mdbook-cmdrun
-```
-
-Run it locally
-```
+# Run the mdBook
 make run
 ```
-Open your browser [here](http://localhost:9999/)
 
-GitHub hosted [version](https://smartcontractkit.github.io/chainlink-testing-framework/overview.html)
+Visit [localhost:9999](http://localhost:9999) to see your local version.

book/README.md

@@ -15,10 +15,10 @@
     - [Configuration](./framework/configuration.md)
     - [Test Configuration](./framework/test_configuration_overrides.md)
     - [Components Persistence](framework/components/state.md)
+    - [Components Cleanup](framework/components/cleanup.md)
     - [Components Caching](framework/components/caching.md)
     - [External Environment](framework/components/external.md)
     - [Secrets]()
-    - [Docker](framework/docker.md)
     - [Observability Stack](framework/observability/observability_stack.md)
       - [Metrics](framework/observability/metrics.md)
       - [Logs](framework/observability/logs.md)
@@ -39,7 +39,7 @@
       - [Chainlink]()
       - [RPC]()
       - [Loki]()
-- [Testing](framework/testing.md)
+- [Testing Maturity Model](framework/testing.md)
   - [Smoke]()
   - [Performance]()
   - [Chaos]()

book/src/SUMMARY.md

@@ -0,0 +1,23 @@
+# Components Cleanup
+
+Managing state is challenging, especially in end-to-end testing, we use [ryuk](https://golang.testcontainers.org/features/garbage_collector/#ryuk) and following simple rules:
+- If `TESTCONTAINERS_RYUK_DISABLED=true`, no cleanup occurs — containers, volumes, and networks remain on your machine.
+
+  Feel free to use `ctf d rm` to remove containers when you are ready.
+- If `TESTCONTAINERS_RYUK_DISABLED` is unset, the test environment will be automatically cleaned up a few seconds after the test completes.
+
+
+Keep in mind that all components are mapped to [static ports](state.md), so without cleanup, only one environment can run at a time.
+
+This design choice simplifies debugging.
+
+A simplified command is available to prune unused volumes, containers, and build caches. Use it when you’re running low on space on your machine.
+```
+ctf d c
+```
+
+<div class="warning">
+
+The framework manages cleanup for both on-chain and off-chain Docker components. However, if your test involves actions like configuring Chainlink jobs, it's best practice to make these actions idempotent, so they can be applied reliably in any environment.
+
+</div>