Skip to content

Commit 8195721

Browse files
authored
all: add flake.nix setup and CLAUDE.md documentation (#6106)
1 parent 8929c43 commit 8195721

File tree

6 files changed

+691
-1
lines changed

6 files changed

+691
-1
lines changed

.gitignore

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,4 +32,12 @@ lcov.info
3232

3333
# Docker volumes and debug logs
3434
.postgres
35-
logfile
35+
logfile
36+
37+
# Nix related files
38+
.direnv
39+
.envrc
40+
.data
41+
42+
# Local claude settings
43+
.claude/settings.local.json

CLAUDE.md

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
Graph Node is a Rust-based decentralized blockchain indexing protocol that enables efficient querying of blockchain data through GraphQL. It's the core component of The Graph protocol, written as a Cargo workspace with multiple crates organized by functionality.
8+
9+
## Essential Development Commands
10+
11+
### Testing Workflow
12+
13+
⚠️ **Only run integration tests when explicitly requested or when changes require full system testing**
14+
15+
Use unit tests for regular development and only run integration tests when:
16+
17+
- Explicitly asked to do so
18+
- Making changes to integration/end-to-end functionality
19+
- Debugging issues that require full system testing
20+
- Preparing releases or major changes
21+
22+
### Unit Tests
23+
24+
Unit tests are inlined with source code.
25+
26+
**Prerequisites:**
27+
1. PostgreSQL running on localhost:5432 (with initialised `graph-test` database)
28+
2. IPFS running on localhost:5001
29+
3. Yarn (v1)
30+
4. Foundry (for smart contract compilation)
31+
5. Environment variable `THEGRAPH_STORE_POSTGRES_DIESEL_URL` set
32+
33+
The environment dependencies and environment setup are operated by the human.
34+
35+
**Running Unit Tests:**
36+
```bash
37+
# REQUIRED: Export database connection string before running unit tests (do this **once** at the beginning of a testing session)
38+
export THEGRAPH_STORE_POSTGRES_DIESEL_URL="postgresql://graph:[email protected]:5432/graph-test"
39+
40+
# Run unit tests (excluding integration tests)
41+
cargo test --workspace --exclude graph-tests
42+
43+
# Run specific tests
44+
cargo test -p graph data_source::common::tests
45+
cargo test <specific_test_name>
46+
```
47+
48+
### Runner Tests (Integration Tests)
49+
50+
**Prerequisites:**
51+
1. PostgreSQL running on localhost:5432 (with initialised `graph-test` database)
52+
2. IPFS running on localhost:5001
53+
3. Yarn (v1)
54+
4. Foundry (for smart contract compilation)
55+
5. Environment variable `THEGRAPH_STORE_POSTGRES_DIESEL_URL` set
56+
57+
**Running Runner Tests:**
58+
```bash
59+
# REQUIRED: Export database connection string before running unit tests (do this **once** at the beginning of a testing session)
60+
export THEGRAPH_STORE_POSTGRES_DIESEL_URL="postgresql://graph:[email protected]:5432/graph-test"
61+
62+
# Run all runner tests
63+
cargo test -p graph-tests --test runner_tests -- --nocapture
64+
65+
# Run specific runner tests
66+
cargo test -p graph-tests --test runner_tests test_name -- --nocapture
67+
```
68+
69+
**Important Notes:**
70+
- Runner tests take moderate time (10-20 seconds)
71+
- Tests automatically reset the database between runs
72+
- Some tests can pass without IPFS, but tests involving file data sources or substreams require it
73+
74+
### Integration Tests
75+
76+
**Prerequisites:**
77+
1. PostgreSQL running on localhost:3011 (with initialised `graph-node` database)
78+
2. IPFS running on localhost:3001
79+
3. Anvil running on localhost:3021
80+
4. Yarn (v1)
81+
5. Foundry (for smart contract compilation)
82+
83+
The environment dependencies and environment setup are operated by the human.
84+
85+
**Running Integration Tests:**
86+
```bash
87+
# Run all integration tests
88+
cargo test -p graph-tests --test integration_tests -- --nocapture
89+
90+
# Run a specific integration test case (e.g., "grafted" test case)
91+
TEST_CASE=grafted cargo test -p graph-tests --test integration_tests -- --nocapture
92+
```
93+
94+
**Important Notes:**
95+
- Integration tests take significant time (several minutes)
96+
- Tests automatically reset the database between runs
97+
- Logs are written to `tests/integration-tests/graph-node.log`
98+
99+
### Code Quality
100+
```bash
101+
# 🚨 MANDATORY: Format all code IMMEDIATELY after any .rs file edit
102+
cargo fmt --all
103+
104+
# 🚨 MANDATORY: Check code for warnings and errors - MUST have zero warnings
105+
cargo check
106+
```
107+
108+
🚨 **CRITICAL REQUIREMENTS for ANY implementation**:
109+
- **🚨 MANDATORY**: `cargo fmt --all` MUST be run before any commit
110+
- **🚨 MANDATORY**: `cargo check` MUST show zero warnings before any commit
111+
- **🚨 MANDATORY**: The unit test suite MUST pass before any commit
112+
113+
Forgetting any of these means you failed to follow instructions. Before any commit or PR, ALL of the above MUST be satisfied! No exceptions!
114+
115+
## High-Level Architecture
116+
117+
### Core Components
118+
- **`graph/`**: Core abstractions, traits, and shared types
119+
- **`node/`**: Main executable and CLI (graphman)
120+
- **`chain/`**: Blockchain-specific adapters (ethereum, near, substreams)
121+
- **`runtime/`**: WebAssembly runtime for subgraph execution
122+
- **`store/`**: PostgreSQL-based storage layer
123+
- **`graphql/`**: GraphQL query execution engine
124+
- **`server/`**: HTTP/WebSocket APIs
125+
126+
### Data Flow
127+
```
128+
Blockchain → Chain Adapter → Block Stream → Trigger Processing → Runtime → Store → GraphQL API
129+
```
130+
131+
1. **Chain Adapters** connect to blockchain nodes and convert data to standardized formats
132+
2. **Block Streams** provide event-driven streaming of blockchain blocks
133+
3. **Trigger Processing** matches blockchain events to subgraph handlers
134+
4. **Runtime** executes subgraph code in WebAssembly sandbox
135+
5. **Store** persists entities with block-level granularity
136+
6. **GraphQL** processes queries and returns results
137+
138+
### Key Abstractions
139+
- **`Blockchain`** trait: Core blockchain interface
140+
- **`Store`** trait: Storage abstraction with read/write variants
141+
- **`RuntimeHost`**: WASM execution environment
142+
- **`TriggerData`**: Standardized blockchain events
143+
- **`EventConsumer`/`EventProducer`**: Component communication
144+
145+
### Architecture Patterns
146+
- **Event-driven**: Components communicate through async streams and channels
147+
- **Trait-based**: Extensive use of traits for abstraction and modularity
148+
- **Async/await**: Tokio-based async runtime throughout
149+
- **Multi-shard**: Database sharding for scalability
150+
- **Sandboxed execution**: WASM runtime with gas metering
151+
152+
## Development Guidelines
153+
154+
### Commit Convention
155+
Use format: `{crate-name}: {description}`
156+
- Single crate: `store: Support 'Or' filters`
157+
- Multiple crates: `core, graphql: Add event source to store`
158+
- All crates: `all: {description}`
159+
160+
### Git Workflow
161+
- Rebase on master (don't merge master into feature branch)
162+
- Keep commits logical and atomic
163+
- Squash commits to clean up history before merging
164+
165+
## Crate Structure
166+
167+
### Core Crates
168+
- **`graph`**: Shared types, traits, and utilities
169+
- **`node`**: Main binary and component wiring
170+
- **`core`**: Business logic and subgraph management
171+
172+
### Blockchain Integration
173+
- **`chain/ethereum`**: Ethereum chain support
174+
- **`chain/near`**: NEAR protocol support
175+
- **`chain/substreams`**: Substreams data source support
176+
177+
### Infrastructure
178+
- **`store/postgres`**: PostgreSQL storage implementation
179+
- **`runtime/wasm`**: WebAssembly runtime and host functions
180+
- **`graphql`**: Query processing and execution
181+
- **`server/`**: HTTP/WebSocket servers
182+
183+
### Key Dependencies
184+
- **`diesel`**: PostgreSQL ORM
185+
- **`tokio`**: Async runtime
186+
- **`tonic`**: gRPC framework
187+
- **`wasmtime`**: WebAssembly runtime
188+
- **`web3`**: Ethereum interaction
189+
190+
## Test Environment Requirements
191+
192+
### Process Compose Setup (Recommended)
193+
194+
The repository includes a process-compose-flake setup that provides native, declarative service management.
195+
196+
Currently, the human is required to operate the service dependencies as illustrated below.
197+
198+
**Unit Tests:**
199+
```bash
200+
# Human: Start PostgreSQL + IPFS for unit tests in a separate terminal
201+
# PostgreSQL: localhost:5432, IPFS: localhost:5001
202+
nix run .#unit
203+
204+
# Claude: Export the database connection string before running unit tests
205+
export THEGRAPH_STORE_POSTGRES_DIESEL_URL="postgresql://graph:[email protected]:5432/graph-test"
206+
207+
# Claude: Run unit tests
208+
cargo test --workspace --exclude graph-tests
209+
```
210+
211+
**Runner Tests:**
212+
```bash
213+
# Human: Start PostgreSQL + IPFS for runner tests in a separate terminal
214+
# PostgreSQL: localhost:5432, IPFS: localhost:5001
215+
nix run .#unit # NOTE: Runner tests are using the same nix services stack as the unit test
216+
217+
# Claude: Export the database connection string before running runner tests
218+
export THEGRAPH_STORE_POSTGRES_DIESEL_URL="postgresql://graph:[email protected]:5432/graph-test"
219+
220+
# Claude: Run runner tests
221+
cargo test -p graph-tests --test runner_tests -- --nocapture
222+
```
223+
224+
**Integration Tests:**
225+
```bash
226+
# Human: Start all services for integration tests in a separate terminal
227+
# PostgreSQL: localhost:3011, IPFS: localhost:3001, Anvil: localhost:3021
228+
nix run .#integration
229+
230+
# Claude: Run integration tests
231+
cargo test -p graph-tests --test integration_tests
232+
```
233+
234+
**Services Configuration:**
235+
The services are configured to use the test suite default ports for unit- and integration tests respectively.
236+
237+
| Service | Unit Tests Port | Integration Tests Port | Database/Config |
238+
|---------|-----------------|------------------------|-----------------|
239+
| PostgreSQL | 5432 | 3011 | `graph-test` / `graph-node` |
240+
| IPFS | 5001 | 3001 | Data in `./.data/unit` or `./.data/integration` |
241+
| Anvil (Ethereum) | - | 3021 | Deterministic test chain |
242+
243+
**Service Configuration:**
244+
The setup combines built-in services-flake services with custom multiService modules:
245+
246+
**Built-in Services:**
247+
- **PostgreSQL**: Uses services-flake's postgres service with a helper function (`mkPostgresConfig`) that provides graph-specific defaults including required extensions.
248+
249+
**Custom Services** (located in `./nix`):
250+
- `ipfs.nix`: IPFS (kubo) with automatic initialization and configurable ports
251+
- `anvil.nix`: Ethereum test chain with deterministic configuration

0 commit comments

Comments
 (0)