Skip to content

Commit 8090240

Browse files
neithanmoneithanmo
authored
Setup local testnet for integration testing (#679)
* feat(testing): Implement local testing environment for indexer services Set up a comprehensive Docker-based testing environment that simulates a production-like setup for indexer services. This includes: - Integration with local-network components (chain, IPFS, graph-node, contracts) - Custom services configuration with proper database migrations - Test scripts to validate service functionality(WIP) - Fix for database schema creation to support cost models and deny lists - Automated service startup with correct dependency ordering(WIP) This environment allows testing our services against real subgraphs and contracts, providing better validation than isolated unit tests. The setup creates a three-tier architecture that mimics production deployment. Tests can now validate payment receipt processing and query routing in a realistic scenario. * fix(testing): Organize containers and configurations into its owne contrib dir * fix(testing): add reload script and container builder script to setup local network testing * fix(testing): Remove unnecessary tests and files * fix(testing): Update justfile and add a Makefile(for future CI) * feat(testing): Add a minimal workflow for integration testing using local network * fix(service): tap-agent start script to use settings from .env variable and remove logging from indexer start script * fix(testing): Fix docker command in makefile * chore(testnet): Add more logs and double check for contracts to be deployed already * fix(testing_setup): Update local network and enable missing services * feat(local_network): Enable the gateway service * fix(indexer-testing): Use same auth token as gateway * fix(testing): Use reasonable values to configure RAV request in tap-agent for testing on local network * chore(testing): Remove old testing script * feat(testing): Add end to end testing for RAV generation * fix(git): Add new items to gitignore * fix(makefile): Avoid triggering errors if no container is active * fix(ci): Enable test execution on push * fix(rav_test): Increase number of receipts to ensure we hit rav generation thresholds * fix(ci): Disable local testnet testing for now * fix(Makefile): Update help and ensure funding script is executed before tests * fix(testing_plan): Remove code example and improve description of testing considerations * chore(local_testnet): Update README and Fix down action in justfile * fix(license): Add license headers * fix(ci): Ignore Dockerfile, docker compose, bash scripts and configuration files * fix(ci): Restore original Dockerfiles for indexer and tap-agent services * fix(ci): Ignore bash scripts in rav_e2e testing * fix(ci): Fix syntax error * chore(test): Reduce timestamp_buffer_secs window chore(testing): Some enhacements and wait for the system to process receipts fix(testing): Adjust some settings to ensure RAV request fix(testing): Generate enough receipts to ensure RAV generation chore(testing): Give enough time for indexer-service and tap-agent to start before deploying gateway * chore(testing): Improve test by checking for metrics after sending receipts * feat(rav_test): Organize better testing code * fix(gateway_service): Add health check after deploying gateway to ensure it is ready before starting tests * fix(justfile): Update test command * fix(make): Remove Makefile and used Justfile instead * fix(setup): Fix setup script docker storage report and ensure we mine some blocks * fix(docs): Update README instruction to setting up local network and running tests * fix(test): Ensure contracts are mined before verification * fix(test): Sent only 3 receipts per batch otherwise they could be discarted * fix(test): Remove premature block mining * chore(test): Change crate name and made it part of the workspace * fix(license): Update path to exclude bash script for testing
1 parent cf7d42d commit 8090240

27 files changed

+7442
-1
lines changed

‎.github/workflows/license_headers_check.yml‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ name: License headers check
22

33
on:
44
push:
5-
branches: [ main, dev ]
5+
branches: [main, dev]
66
pull_request:
77
workflow_dispatch:
88

@@ -31,4 +31,11 @@ jobs:
3131
-ignore 'migrations/*.sql' \
3232
-ignore 'crates/dips/src/proto/*' \
3333
-ignore 'crates/dips/proto/*' \
34+
-ignore 'contrib/**/*' \
35+
-ignore '*.sh' \
36+
-ignore 'Dockerfile' \
37+
-ignore '*.toml' \
38+
-ignore '*.yml' \
39+
-ignore '*.yaml' \
40+
-ignore 'integration-tests/*.sh' \
3441
.

‎.github/workflows/tap_integration_test.yml.disable‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: Test GraphTally Protocol
2+
on:
3+
push:
4+
# branches: [main]
5+
pull_request:
6+
branches: [main]
7+
8+
jobs:
9+
test:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- name: Checkout code
13+
uses: actions/checkout@v3
14+
15+
- name: Set up Docker
16+
uses: docker/setup-buildx-action@v2
17+
18+
- name: Set up Docker cache
19+
uses: docker/setup-buildx-action@v2
20+
with:
21+
install: true
22+
23+
- name: Run make setup
24+
run: make setup
25+
26+
- name: Run tests
27+
run: make rav_tests
28+
29+
- name: Tear down containers
30+
if: always()
31+
run: make down

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,7 @@ indexer.toml
1616
crates/dips/node_modules/
1717
crates/dips/generated/
1818
crates/dips/npm-debug.log*
19+
contrib/local-network
20+
node_modules
21+
target/
22+
*.code-workspace

‎Cargo.lock‎

Lines changed: 17 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ members = [
1111
"crates/tap-agent",
1212
"crates/test-assets",
1313
"crates/watcher",
14+
"integration-tests",
1415
]
1516
resolver = "2"
1617

‎contrib/README.md‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Developer Setup Guide
2+
3+
This guide provides the step to set up a development workflow for the indexer-service project testing.
4+
Including how to build and deploy the required containers, and how to use hot-reload for faster dev iterations.
5+
6+
### Note
7+
8+
The current implementation leverages local-network for the common services and uses its testing configuration.
9+
10+
## Available Commands
11+
12+
We provide the following Make/Just commands to streamline your development workflow:
13+
14+
- `just setup` - Full setup of all services, dependencies and binary compilation
15+
- `just reload` - Rebuild Rust binaries and restart services after code changes
16+
- `just logs` - Watch log output from all services
17+
- `just down` - Stop all services
18+
- `just test-local` - Run integration tests against local services
19+
20+
## Initial Setup
21+
22+
To get started with development:
23+
24+
1. Clone the repository
25+
2. Run the full setup to initialize all services:
26+
27+
```bash
28+
just setup
29+
```
30+
31+
This will:
32+
33+
- Clone the local-network repo if needed
34+
- Start core infrastructure services
35+
- Deploy contracts and required services
36+
- Set up and start the indexer and tap-agent services
37+
38+
## Development Workflow
39+
40+
After the initial setup, you can use the fast development workflow:
41+
42+
1. Make changes to the Rust code in `crates/`
43+
2. Run the reload command to rebuild and restart indexer and tap-agent services:
44+
45+
```bash
46+
just reload
47+
```
48+
49+
This workflow is much faster because:
50+
51+
- It only recompiles the Rust code
52+
- It doesn't rebuild Docker containers
53+
- It restarts only the necessary services
54+
55+
The `reload` command will automatically show the logs after restarting, so you can see if your changes are working properly.
56+
57+
## How It Works
58+
59+
The development workflow uses volume mounts to avoid rebuilding containers:
60+
61+
1. A base Docker image with all dependencies is created once
62+
2. Your code is compiled locally on your machine
63+
3. The compiled binaries are mounted into the containers
64+
4. Services are restarted with the new binaries
65+
66+
This approach avoids the time-consuming container rebuild process while still maintaining a consistent containerized environment.
67+
68+
## Viewing Logs
69+
70+
To monitor the services during development:
71+
72+
```bash
73+
just logs
74+
```
75+
76+
This will show a live stream of logs from all services, which is useful for debugging.
77+
78+
## Running Tests
79+
80+
To run integration tests against your local environment:
81+
82+
```bash
83+
just local-test
84+
```
85+
86+
This is currently a work in progress, and additional testing strategies are still being defined and implemented.
87+
88+
## Stopping Services
89+
90+
When you're done working:
91+
92+
```bash
93+
just down
94+
```
95+
96+
This will stop and remove all the containers defined in the docker-compose file.

‎contrib/base/Dockerfile‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
FROM debian:bookworm-slim
2+
3+
RUN apt-get update && apt-get install -y --no-install-recommends \
4+
openssl ca-certificates protobuf-compiler postgresql-client curl \
5+
jq \
6+
&& rm -rf /var/lib/apt/lists/*

‎contrib/docker-compose.dev.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
services:
2+
indexer-service:
3+
image: indexer-base:latest
4+
container_name: indexer-service
5+
volumes:
6+
- ../target/release/indexer-service-rs:/usr/local/bin/indexer-service-rs
7+
- ./indexer-service/start.sh:/usr/local/bin/start.sh
8+
- ./indexer-service/config.toml:/opt/config/config.toml
9+
- ../local-network/contracts.json:/opt/contracts.json:ro
10+
- ../local-network/.env:/opt/.env:ro
11+
- ../migrations:/opt/migrations:ro
12+
entrypoint: ["/usr/local/bin/start.sh"]
13+
environment:
14+
- RUST_BACKTRACE=1
15+
- RUST_LOG=debug
16+
ports:
17+
- "7601:7601"
18+
networks:
19+
- local-network
20+
healthcheck:
21+
test: ["CMD", "curl", "-f", "http://localhost:7601/"]
22+
interval: 5s
23+
timeout: 3s
24+
retries: 10
25+
start_period: 10s
26+
27+
tap-agent:
28+
image: indexer-base:latest # Pre-built base image with dependencies
29+
container_name: tap-agent
30+
depends_on:
31+
indexer-service:
32+
condition: service_healthy
33+
volumes:
34+
- ../target/release/indexer-tap-agent:/usr/local/bin/indexer-tap-agent
35+
- ./tap-agent/start.sh:/usr/local/bin/start.sh
36+
- ./tap-agent:/opt/config:ro
37+
- ./local-network/.env:/opt/.env:ro
38+
- ./local-network/contracts.json:/opt/contracts.json:ro
39+
- ../migrations:/opt/migrations:ro
40+
entrypoint: ["/bin/bash", "-c", "/opt/config/start.sh"]
41+
environment:
42+
- RUST_BACKTRACE=1
43+
- RUST_LOG=debug
44+
ports:
45+
# to expose the metrics port
46+
- "7300:7300"
47+
healthcheck:
48+
test: ["CMD", "curl", "-f", "http://localhost:7300/metrics"]
49+
interval: 5s
50+
timeout: 3s
51+
retries: 10
52+
start_period: 10s
53+
54+
networks:
55+
local-network:
56+
external: true
57+
name: local-network_default

‎contrib/docker-compose.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
services:
2+
indexer-service:
3+
build:
4+
context: .. # Go up one level to the root directory
5+
dockerfile: ./contrib/indexer-service/Dockerfile
6+
container_name: indexer-service
7+
volumes:
8+
- ./indexer-service:/opt/config:ro # From contrib dir to indexer-service dir
9+
- ./local-network/contracts.json:/opt/contracts.json:ro
10+
- ./local-network/.env:/opt/.env:ro
11+
- ../migrations:/opt/migrations:ro
12+
entrypoint: ["/bin/bash", "-c", "/opt/config/start.sh"]
13+
environment:
14+
- RUST_BACKTRACE=1
15+
- RUST_LOG=debug
16+
ports:
17+
- "7601:7601"
18+
networks:
19+
- local-network
20+
healthcheck:
21+
test: ["CMD", "curl", "-f", "http://localhost:7601/"]
22+
interval: 5s
23+
timeout: 3s
24+
retries: 10
25+
start_period: 10s
26+
27+
tap-agent:
28+
build:
29+
context: ..
30+
dockerfile: ./contrib/tap-agent/Dockerfile
31+
container_name: tap-agent
32+
depends_on:
33+
indexer-service:
34+
condition: service_healthy
35+
volumes:
36+
- ./tap-agent:/opt/config:ro # From contrib dir to tap-agent dir
37+
- ./local-network/.env:/opt/.env:ro
38+
- ./local-network/contracts.json:/opt/contracts.json:ro
39+
- ../migrations:/opt/migrations:ro
40+
entrypoint: ["/bin/bash", "-c", "/opt/config/start.sh"]
41+
environment:
42+
- RUST_BACKTRACE=1
43+
- RUST_LOG=debug
44+
ports:
45+
# to expose the metrics port
46+
- "7300:7300"
47+
healthcheck:
48+
test: ["CMD", "curl", "-f", "http://localhost:7300/metrics"]
49+
interval: 5s
50+
timeout: 3s
51+
retries: 10
52+
start_period: 10s
53+
54+
networks:
55+
local-network:
56+
external: true
57+
name: local-network_default

‎contrib/indexer-service/Dockerfile‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
FROM rust:1.81-bookworm as build
2+
WORKDIR /root
3+
# Copy from the root project directory (two levels up)
4+
COPY ../../ .
5+
# Force SQLx to use the offline mode to statically check the database queries against
6+
# the prepared files in the `.sqlx` directory.
7+
#
8+
ENV SQLX_OFFLINE=true
9+
10+
RUN apt-get update && apt-get install -y --no-install-recommends \
11+
protobuf-compiler && rm -rf /var/lib/apt/lists/*
12+
13+
RUN cargo build --release --bin indexer-service-rs
14+
15+
########################################################################################
16+
17+
FROM debian:bookworm-slim
18+
19+
RUN apt-get update && apt-get install -y --no-install-recommends \
20+
openssl ca-certificates protobuf-compiler postgresql-client curl \
21+
jq \
22+
&& rm -rf /var/lib/apt/lists/*
23+
24+
COPY --from=build /root/target/release/indexer-service-rs /usr/local/bin/indexer-service-rs
25+
26+
# Copy our start script into the image
27+
COPY contrib/indexer-service/start.sh /usr/local/bin/start.sh
28+
COPY contrib/indexer-service/config.toml /opt/config/config.toml
29+
30+
RUN chmod +x /usr/local/bin/start.sh
31+
32+
ENTRYPOINT [ "/usr/local/bin/start.sh" ]

0 commit comments

Comments
 (0)