Skip to content

Commit 2df5815

Browse files
authored
feat: simplify devnet setup (#109)
This PR updates the devnet setup we have in our repo: - Removes the metrics stack, which was integrated into lean-quickstart in blockblaz/lean-quickstart#109 - Updates the `run-devnet` target to use the just built Docker tag for the devnet - Adds a `make fmt` target for formatting the code - Update the documentation with a list of Makefile targets and removing mentions of our metrics stack
1 parent e1d9c0c commit 2df5815

File tree

12 files changed

+96
-4456
lines changed

12 files changed

+96
-4456
lines changed

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,3 +30,9 @@ target
3030

3131
# Used for generating test vectors
3232
/leanSpec
33+
34+
# Personal Claude Code overrides (not shared with team)
35+
CLAUDE.local.md
36+
37+
# Log output by make run-devnet
38+
devnet.log

CLAUDE.md

Lines changed: 62 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -13,20 +13,28 @@ Not to be confused with Ethereum consensus clients AKA Beacon Chain clients AKA
1313

1414
```
1515
bin/ethlambda/ # Entry point, CLI, orchestration
16+
└─ src/version.rs # Build-time version info (vergen-git2)
1617
crates/
1718
blockchain/ # State machine actor (GenServer pattern)
1819
├─ src/lib.rs # BlockChain actor, tick events, validator duties
1920
├─ src/store.rs # Fork choice store, block/attestation processing
21+
├─ src/key_manager.rs # Validator key management and signing
22+
├─ src/metrics.rs # Blockchain-level Prometheus metrics
2023
├─ fork_choice/ # LMD GHOST implementation (3SF-mini)
2124
└─ state_transition/ # STF: process_slots, process_block, attestations
25+
└─ src/metrics.rs # State transition timing + counters
2226
common/
2327
├─ types/ # Core types (State, Block, Attestation, Checkpoint)
2428
├─ crypto/ # XMSS aggregation (leansig wrapper)
25-
└─ metrics/ # Prometheus metrics
29+
└─ metrics/ # Prometheus re-exports, TimingGuard, gather utilities
2630
net/
2731
├─ p2p/ # libp2p: gossipsub + req-resp (Status, BlocksByRoot)
28-
└─ rpc/ # Axum HTTP endpoints (/lean/v0/* and /metrics)
32+
│ ├─ src/gossipsub/ # Topic encoding, message handling
33+
│ ├─ src/req_resp/ # Request/response codec and handlers
34+
│ └─ src/metrics.rs # Peer connection/disconnection tracking
35+
└─ rpc/ # Axum HTTP: /lean/v0/{states,checkpoints,health} + /metrics
2936
storage/ # RocksDB backend, in-memory for tests
37+
└─ src/api/ # StorageBackend trait + Table enum
3038
```
3139

3240
## Key Architecture Patterns
@@ -64,7 +72,7 @@ Fork choice head update
6472

6573
### Before Committing
6674
```bash
67-
cargo fmt # Format code
75+
make fmt # Format code (cargo fmt --all)
6876
make lint # Clippy with -D warnings
6977
make test # All tests + forkchoice (with skip-signature-verification)
7078
```
@@ -73,6 +81,8 @@ make test # All tests + forkchoice (with skip
7381
```bash
7482
.claude/skills/test-pr-devnet/scripts/test-branch.sh # Test branch in multi-client devnet
7583
rm -rf leanSpec && make leanSpec/fixtures # Regenerate test fixtures (requires uv)
84+
make docker-build # Build Docker image (DOCKER_TAG=local)
85+
make run-devnet # Run local devnet with lean-quickstart
7686
```
7787

7888
### Testing with Local Devnet
@@ -95,10 +105,10 @@ let byte: u8 = code.into();
95105
### Ownership for Large Structures
96106
```rust
97107
// Prefer taking ownership to avoid cloning large data (signatures ~3KB)
98-
pub fn consume_signed_block(signed_block: SignedBlockWithAttestation) { ... }
108+
pub fn insert_signed_block(&mut self, root: H256, signed_block: SignedBlockWithAttestation) { ... }
99109

100110
// Add .clone() at call site if needed - makes cost explicit
101-
store.insert_signed_block(root, signed_block.clone());
111+
store.insert_signed_block(block_root, signed_block.clone());
102112
```
103113

104114
### Formatting Patterns
@@ -157,12 +167,34 @@ let result = operation()
157167
.map_err(|err| CustomError::from(err))?;
158168
```
159169

160-
### Metrics (RAII Pattern)
170+
### Metrics Patterns
171+
172+
**Registration with `LazyLock`:**
173+
```rust
174+
// Module-scoped statics (preferred for state_transition metrics)
175+
static LEAN_STATE_TRANSITION_TIME_SECONDS: LazyLock<Histogram> = LazyLock::new(|| {
176+
register_histogram!("lean_metric_name", "Description", vec![...]).unwrap()
177+
});
178+
179+
// Function-scoped statics (used in blockchain metrics)
180+
pub fn update_head_slot(slot: u64) {
181+
static LEAN_HEAD_SLOT: LazyLock<IntGauge> = LazyLock::new(|| {
182+
register_int_gauge!("lean_head_slot", "Latest slot").unwrap()
183+
});
184+
LEAN_HEAD_SLOT.set(slot.try_into().unwrap());
185+
}
186+
```
187+
188+
**RAII timing guard (auto-observes duration on drop):**
161189
```rust
162-
// Timing guard automatically observes duration on drop
163190
let _timing = metrics::time_state_transition();
164191
```
165192

193+
**All metrics use `ethlambda_metrics::*` re-exports** — the `ethlambda-metrics` crate re-exports
194+
prometheus types (`IntGauge`, `IntCounter`, `Histogram`, etc.) and provides `TimingGuard` + `gather_default_metrics()`.
195+
196+
**Naming convention:** All metrics use `lean_` prefix (e.g., `lean_head_slot`, `lean_state_transition_time_seconds`).
197+
166198
### Logging Patterns
167199

168200
**Use tracing shorthand syntax for cleaner logs:**
@@ -281,8 +313,27 @@ cargo test -p ethlambda-blockchain --features skip-signature-verification --test
281313
- Crypto tests marked `#[ignore]` (slow leanVM operations)
282314

283315
### Storage Architecture
284-
- Genesis block has no signatures - stored in Blocks table only, not BlockSignatures
285-
- All other blocks must have entries in both tables
316+
- Blocks are split into three tables: `BlockHeaders`, `BlockBodies`, `BlockSignatures`
317+
- Genesis/anchor blocks have empty bodies (detected via `EMPTY_BODY_ROOT`) — no entry in `BlockBodies`
318+
- Genesis block has no signatures — no entry in `BlockSignatures`
319+
- All other blocks must have entries in all three tables
320+
- `LiveChain` table provides fast `(slot||root) → parent_root` index for fork choice
321+
- Storage uses trait-based API: `StorageBackend` → `StorageReadView` (reads) + `StorageWriteBatch` (atomic writes)
322+
323+
### Storage Tables (10)
324+
325+
| Table | Key → Value | Purpose |
326+
|-------|-------------|---------|
327+
| `BlockHeaders` | H256 → BlockHeader | Block headers by root |
328+
| `BlockBodies` | H256 → BlockBody | Block bodies (empty for genesis) |
329+
| `BlockSignatures` | H256 → BlockSignaturesWithAttestation | Signatures (absent for genesis) |
330+
| `States` | H256 → State | Beacon states by root |
331+
| `LatestKnownAttestations` | u64 → AttestationData | Fork-choice-active attestations |
332+
| `LatestNewAttestations` | u64 → AttestationData | Pending (pre-promotion) attestations |
333+
| `GossipSignatures` | SignatureKey → ValidatorSignature | Individual validator signatures |
334+
| `AggregatedPayloads` | SignatureKey → Vec\<AggregatedSignatureProof\> | Aggregated proofs |
335+
| `Metadata` | string → various | Store state (head, config, checkpoints) |
336+
| `LiveChain` | (slot\|\|root) → parent\_root | Fast fork choice traversal index |
286337

287338
### State Root Computation
288339
- Always computed via `tree_hash_root()` after full state transition
@@ -304,6 +355,7 @@ cargo test -p ethlambda-blockchain --features skip-signature-verification --test
304355
- `tree_hash`: Merkle tree hashing
305356
- `spawned-concurrency`: Actor model
306357
- `libp2p`: P2P networking (custom LambdaClass fork)
358+
- `vergen-git2`: Build-time git commit/branch info embedded in binary
307359

308360
**Storage:**
309361
- `rocksdb`: Persistent backend
@@ -313,6 +365,7 @@ cargo test -p ethlambda-blockchain --features skip-signature-verification --test
313365

314366
**Specs:** `leanSpec/src/lean_spec/` (Python reference implementation)
315367
**Devnet:** `lean-quickstart` (github.com/blockblaz/lean-quickstart)
368+
**Releases:** See `RELEASE.md` for release process documentation
316369

317370
## Other implementations
318371

Makefile

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,11 @@
1-
.PHONY: help lint docker-build run-devnet test
1+
.PHONY: help fmt lint docker-build run-devnet test
22

33
help: ## 📚 Show help for each of the Makefile recipes
44
@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
55

6+
fmt: ## 🎨 Format all code using rustfmt
7+
cargo fmt --all
8+
69
lint: ## 🔍 Run clippy on all workspace crates
710
cargo clippy --workspace --all-targets -- -D warnings
811

@@ -20,6 +23,7 @@ docker-build: ## 🐳 Build the Docker image
2023
--build-arg GIT_COMMIT=$(GIT_COMMIT) \
2124
--build-arg GIT_BRANCH=$(GIT_BRANCH) \
2225
-t ghcr.io/lambdaclass/ethlambda:$(DOCKER_TAG) .
26+
@echo
2327

2428
LEAN_SPEC_COMMIT_HASH:=4edcf7bc9271e6a70ded8aff17710d68beac4266
2529

@@ -33,12 +37,14 @@ leanSpec/fixtures: leanSpec
3337
lean-quickstart:
3438
git clone https://github.com/blockblaz/lean-quickstart.git --depth 1 --single-branch
3539

36-
37-
# TODO: start metrics too
3840
run-devnet: docker-build lean-quickstart ## 🚀 Run a local devnet using lean-quickstart
39-
# Go to lean-quickstart/local-devnet/genesis/validator-config.yaml to modify
40-
# the validator configuration for the local devnet.
41-
# NOTE: to run the local image of ethlambda, make sure to set the image tag
42-
# in lean-quickstart/client-cmds/ethlambda-cmd.sh to "ghcr.io/lambdaclass/ethlambda:local"
43-
cd lean-quickstart \
44-
&& NETWORK_DIR=local-devnet ./spin-node.sh --node all --generateGenesis --metrics
41+
@echo "Starting local devnet with ethlambda client (\"$(DOCKER_TAG)\" tag). Logs will be dumped in devnet.log, and metrics served in http://localhost:3000"
42+
@echo
43+
@echo "Devnet will be using the current configuration. For custom configurations, modify lean-quickstart/local-devnet/genesis/validator-config.yaml and restart the devnet."
44+
@echo
45+
@# Use temp file instead of sed -i for macOS/GNU portability
46+
@sed 's|ghcr.io/lambdaclass/ethlambda:[^ ]*|ghcr.io/lambdaclass/ethlambda:$(DOCKER_TAG)|' lean-quickstart/client-cmds/ethlambda-cmd.sh > lean-quickstart/client-cmds/ethlambda-cmd.sh.tmp \
47+
&& mv lean-quickstart/client-cmds/ethlambda-cmd.sh.tmp lean-quickstart/client-cmds/ethlambda-cmd.sh
48+
@echo "Starting local devnet. Press Ctrl+C to stop all nodes."
49+
@cd lean-quickstart \
50+
&& NETWORK_DIR=local-devnet ./spin-node.sh --node all --generateGenesis --metrics > ../devnet.log 2>&1

README.md

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,17 @@ Minimalist, fast and modular implementation of the Lean Ethereum client written
44

55
## Getting started
66

7-
We use `cargo` as our build system. To build and run the client, simply run:
7+
We use `cargo` as our build system, but prefer `make` as a convenient wrapper for common tasks. These are some common targets:
88

99
```sh
10-
cargo run
10+
# Formats all code
11+
make fmt
12+
# Checks and lints the code
13+
make lint
14+
# Runs all tests
15+
make test
16+
# Builds a docker image tagged as "ghcr.io/lambdaclass/ethlambda:local"
17+
make docker-build DOCKER_TAG=local
1118
```
1219

1320
Run `make help` or take a look at our [`Makefile`](./Makefile) for other useful commands.
@@ -21,9 +28,11 @@ To run a local devnet with multiple clients using [lean-quickstart](https://gith
2128
make run-devnet
2229
```
2330

24-
This generates fresh genesis files and starts all three clients with metrics enabled.
31+
This generates fresh genesis files and starts all configured clients with metrics enabled.
2532
Press `Ctrl+C` to stop all nodes.
2633

34+
For custom devnet configurations, go to `lean-quickstart/local-devnet/genesis/validator-config.yaml` and edit the file before running the command above. See `lean-quickstart`'s documentation for more details on how to configure the devnet.
35+
2736
## Philosophy
2837

2938
Many long-established clients accumulate bloat over time. This often occurs due to the need to support legacy features for existing users or through attempts to implement overly ambitious software. The result is often complex, difficult-to-maintain, and error-prone systems.

docs/metrics.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
We collect various metrics and serve them via a Prometheus-compatible HTTP endpoint at `http://<metrics_address>:<metrics_port>/metrics` (default: `http://127.0.0.1:5054/metrics`).
44

5-
We provide a ready-to-use Grafana + Prometheus monitoring stack in the [`metrics/`](../metrics/) directory. It includes pre-configured dashboards from the [leanMetrics](https://github.com/leanEthereum/leanMetrics) repository for visualizing the metrics described below. See the [metrics README](../metrics/README.md) for setup instructions.
5+
A ready-to-use Grafana + Prometheus monitoring stack with pre-configured [leanMetrics](https://github.com/leanEthereum/leanMetrics) dashboards is available in [lean-quickstart](https://github.com/blockblaz/lean-quickstart).
66

77
The exposed metrics follow [the leanMetrics specification](https://github.com/leanEthereum/leanMetrics/blob/3b32b300cca5ed7a7a2b3f142273fae9dbc171bf/metrics.md), with some metrics not yet implemented. We have a full list of implemented metrics below, with a checkbox indicating whether each metric is currently supported or not.
88

metrics/README.md

Lines changed: 0 additions & 57 deletions
This file was deleted.

metrics/docker-compose-metrics.yaml

Lines changed: 0 additions & 51 deletions
This file was deleted.

0 commit comments

Comments
 (0)