Skip to content

Commit 9493a04

Browse files
committed
Split locks
1 parent e60ad68 commit 9493a04

File tree

6 files changed

+195
-116
lines changed

6 files changed

+195
-116
lines changed

contracts/lib/risc0-ethereum

Submodule risc0-ethereum updated 131 files

crates/CLAUDE.md

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Build Commands
6+
7+
### Rust Workspace
8+
- **Build all crates**: `cargo build` (from `/crates` directory)
9+
- **Build specific crate**: `cargo build --manifest-path ./crates/[crate-name]/Cargo.toml`
10+
- **Build with release optimization**: `cargo build --release`
11+
12+
### Batcher
13+
- **Build**: `cargo build --manifest-path ./crates/batcher/Cargo.toml --release`
14+
- **Run**: `cargo run --manifest-path ./crates/batcher/Cargo.toml --release -- --config ./config-files/config-batcher.yaml --env-file ./crates/batcher/.env`
15+
- **Start locally**: `make batcher_start_local`
16+
17+
### CLI
18+
- **Build**: `cd crates/cli && cargo build --release`
19+
- **Install**: `cargo install --path crates/cli`
20+
- **Install script**: `./crates/cli/install_aligned.sh`
21+
22+
### SDK
23+
- **Build**: `cargo build --manifest-path ./crates/sdk/Cargo.toml`
24+
- **Test**: `cargo test --manifest-path ./crates/sdk/Cargo.toml`
25+
26+
## Testing Commands
27+
28+
### Rust Tests
29+
- **Run all tests**: `cargo test` (from `/crates` directory)
30+
- **Run specific crate tests**: `cargo test --manifest-path ./crates/[crate-name]/Cargo.toml`
31+
- **Run with release mode**: `cargo test --release`
32+
33+
### Go Tests
34+
- **Run all Go tests**: `go test ./... -timeout 15m`
35+
- **Run retry tests**: `cd core/ && go test -v -timeout 15m`
36+
37+
### FFI Tests
38+
- **SP1 Rust FFI**: `cd operator/sp1/lib && RUST_MIN_STACK=83886080 cargo test --release`
39+
- **RISC Zero Rust FFI**: `cd operator/risc_zero/lib && cargo test --release`
40+
- **Merkle Tree FFI**: `cd operator/merkle_tree/lib && RUST_MIN_STACK=83886080 cargo test --release`
41+
42+
## Linting Commands
43+
44+
### Solidity Contracts
45+
- **Lint contracts**: `cd contracts && npm run lint:sol`
46+
47+
### Rust (via Makefile targets)
48+
- Check individual crate formatting: `cargo fmt --check --manifest-path ./crates/[crate-name]/Cargo.toml`
49+
- Check individual crate linting: `cargo clippy --manifest-path ./crates/[crate-name]/Cargo.toml`
50+
51+
## Common Development Commands
52+
53+
### Dependencies
54+
- **Install all dependencies**: `make deps`
55+
- **Install Go dependencies**: `make go_deps`
56+
- **Initialize submodules**: `make submodules`
57+
58+
### Development Environment
59+
- **Start Anvil**: `make anvil_start`
60+
- **Start full local environment**: `make setup_local_aligned_all`
61+
- **Build all FFIs**: `make build_all_ffi`
62+
63+
### Proof Submission
64+
- **Send SP1 proof**: `make batcher_send_sp1_task RPC_URL=http://localhost:8545 NETWORK=devnet`
65+
- **Send RISC0 proof**: `make batcher_send_risc0_task RPC_URL=http://localhost:8545 NETWORK=devnet`
66+
- **Send Gnark proofs**: `make batcher_send_gnark_plonk_bn254_task RPC_URL=http://localhost:8545 NETWORK=devnet`
67+
68+
## Architecture Overview
69+
70+
### Core Components
71+
72+
**Aligned Layer** is a verification layer for zero-knowledge proofs built on EigenLayer. The system consists of several key components:
73+
74+
1. **Batcher** (`crates/batcher/`): Aggregates multiple proofs into batches for efficient verification
75+
- Listens for WebSocket connections from clients
76+
- Collects verification data and batches them based on time/size thresholds
77+
- Submits batches to the verification layer
78+
79+
2. **SDK** (`crates/sdk/`): Provides client libraries for interacting with Aligned Layer
80+
- **Verification Layer**: Core verification functionality
81+
- **Aggregation Layer**: Handles proof aggregation modes
82+
- **Communication**: Protocol implementations for client-server communication
83+
- **Ethereum Integration**: Smart contract interfaces and utilities
84+
85+
3. **CLI** (`crates/cli/`): Command-line interface for submitting proofs and interacting with the system
86+
- Proof submission with various proving systems (SP1, RISC0, Gnark, Circom)
87+
- Balance queries and verification status checks
88+
- Batch verification data handling
89+
90+
4. **Task Sender** (`crates/task-sender/`): Utility for load testing and automated proof submission
91+
- Wallet generation and funding
92+
- Infinite proof submission with configurable parameters
93+
- Connection testing utilities
94+
95+
### Supported Proving Systems
96+
97+
The system supports multiple zero-knowledge proving systems:
98+
- **SP1**: Succinct's zkVM proving system
99+
- **RISC Zero**: General-purpose zkVM
100+
- **Gnark**: Groth16 and PLONK protocols (BN254, BLS12-381)
101+
- **Circom**: Circuit compiler with Groth16 backend
102+
103+
### Key Architectural Patterns
104+
105+
1. **Modular Design**: Each component (batcher, SDK, CLI) is a separate crate with clear boundaries
106+
2. **Async/Await**: Heavy use of Tokio for asynchronous operations
107+
3. **FFI Integration**: Foreign function interfaces for integrating with Go-based verifiers
108+
4. **EigenLayer Integration**: Built as an AVS (Actively Validated Service) on EigenLayer
109+
5. **Multi-Network Support**: Configurable for different networks (devnet, testnet, mainnet)
110+
111+
### Development Workflow
112+
113+
1. **Local Development**: Use `make anvil_start` to start local blockchain
114+
2. **Component Testing**: Each crate can be built and tested independently
115+
3. **Integration Testing**: Full system testing using Docker compose or Makefile targets
116+
4. **Proof Generation**: Scripts in `scripts/test_files/` for generating test proofs
117+
118+
### Configuration Management
119+
120+
- **YAML Configuration**: Primary configuration files in `config-files/`
121+
- **Environment Variables**: `.env` files for sensitive configuration
122+
- **Network-Specific Config**: Separate configurations for different networks
123+
- **Makefile Parameters**: Extensive use of Make variables for configuration

crates/batcher/src/connection.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ use tokio_tungstenite::{
1818
pub(crate) type WsMessageSink = Arc<RwLock<SplitSink<WebSocketStream<TcpStream>, Message>>>;
1919

2020
pub(crate) async fn send_batch_inclusion_data_responses(
21-
finalized_batch: Vec<BatchQueueEntry>,
21+
finalized_batch: &[BatchQueueEntry],
2222
batch_merkle_tree: &MerkleTree<VerificationCommitmentBatch>,
2323
) -> Result<(), BatcherError> {
2424
// Finalized_batch is ordered as the PriorityQueue, ordered by: ascending max_fee && if max_fee is equal, by descending nonce.

crates/batcher/src/lib.rs

Lines changed: 4 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@ pub struct Batcher {
100100
/// The general business rule is:
101101
/// - User processing can be done in parallel unless a batch creation is happening
102102
/// - Batch creation needs to be able to change all the states, so all processing
103-
/// needs to be stopped, and all user_states locks need to be taken
103+
/// needs to be stopped, and all user_states locks need to be taken
104104
batch_state: Mutex<BatchState>,
105105
user_states: DashMap<Address, Arc<Mutex<UserState>>>,
106106
/// When posting a task, this is taken as a write to stop new threads to update
@@ -1317,68 +1317,9 @@ impl Batcher {
13171317
finalized_batch.len()
13181318
);
13191319

1320-
// PHASE 1.5: Update user states immediately after batch extraction to make the operation atomic
1321-
// We assume the batch posting will be successful, so we update user states now
1322-
if let Err(e) = self.update_user_states_after_batch_extraction(&batch_state_lock).await {
1323-
error!("Failed to update user states after batch extraction: {:?}", e);
1324-
// We could potentially put the batch back in the queue here if needed
1325-
*batch_posting = false;
1326-
return None;
1327-
}
1328-
13291320
Some(finalized_batch)
13301321
}
13311322

1332-
/// Updates user states after successful batch submission.
1333-
/// This function should be called only AFTER the submission was confirmed onchain.
1334-
/// Note: Proofs were already removed from the queue during the extraction phase.
1335-
async fn update_user_states_after_batch_submission(&self) -> Result<(), BatcherError> {
1336-
info!("Updating user states after batch submission...");
1337-
let batch_state_lock = self.batch_state.lock().await;
1338-
1339-
// Calculate the new user_states based on the current queue (proofs already removed)
1340-
let new_user_states = self.calculate_new_user_states_data(&batch_state_lock.batch_queue);
1341-
1342-
let user_addresses: Vec<Address> =
1343-
self.user_states.iter().map(|entry| *entry.key()).collect();
1344-
let default_value = (0, U256::MAX, U256::zero());
1345-
for addr in user_addresses.iter() {
1346-
let (proof_count, max_fee_limit, total_fees_in_queue) =
1347-
new_user_states.get(addr).unwrap_or(&default_value);
1348-
1349-
// FIXME: The case where a the update functions return `None` can only happen when the user was not found
1350-
// in the `user_states` map should not really happen here, but doing this check so that we don't unwrap.
1351-
// Once https://github.com/yetanotherco/aligned_layer/issues/1046 is done we could return a more
1352-
// informative error.
1353-
1354-
// Now we update the user states related to the batch (proof count in batch and min fee in batch)
1355-
{
1356-
let user_state = self.user_states.get(addr);
1357-
match user_state {
1358-
Some(user_state) => {
1359-
let mut user_state_guard = user_state.lock().await;
1360-
user_state_guard.proofs_in_batch = *proof_count;
1361-
user_state_guard.last_max_fee_limit = *max_fee_limit;
1362-
user_state_guard.total_fees_in_queue = *total_fees_in_queue;
1363-
}
1364-
None => {
1365-
return Err(BatcherError::QueueRemoveError(
1366-
"Could not update user state".into(),
1367-
));
1368-
}
1369-
}
1370-
}
1371-
}
1372-
1373-
// Update metrics
1374-
let queue_len = batch_state_lock.batch_queue.len();
1375-
let queue_size_bytes = calculate_batch_size(&batch_state_lock.batch_queue)?;
1376-
1377-
self.metrics
1378-
.update_queue_metrics(queue_len as i64, queue_size_bytes as i64);
1379-
1380-
Ok(())
1381-
}
13821323

13831324
/// Updates user states immediately after batch extraction to make the operation atomic.
13841325
/// This function should be called right after extracting proofs from the queue.
@@ -1484,7 +1425,7 @@ impl Batcher {
14841425
/// NOTE: Nonce ordering is preserved by the priority queue's eviction order:
14851426
/// - Lower fees get evicted first
14861427
/// - For same fees, higher nonces get evicted first
1487-
/// This ensures we never have nonce N+1 without nonce N in the queue.
1428+
/// This ensures we never have nonce N+1 without nonce N in the queue.
14881429
async fn restore_proofs_after_batch_failure(&self, failed_batch: &[BatchQueueEntry]) {
14891430
info!("Restoring {} proofs to queue after batch failure", failed_batch.len());
14901431

@@ -1651,7 +1592,7 @@ impl Batcher {
16511592
&batch_bytes,
16521593
&batch_merkle_tree.root,
16531594
leaves,
1654-
&finalized_batch,
1595+
finalized_batch,
16551596
gas_price,
16561597
)
16571598
.await
@@ -1686,7 +1627,7 @@ impl Batcher {
16861627
// User states were also already updated atomically during extraction
16871628

16881629
// Clean up user states for users who had proofs in this batch but now have no proofs left
1689-
self.cleanup_user_states_after_successful_submission(&finalized_batch);
1630+
self.cleanup_user_states_after_successful_submission(finalized_batch);
16901631

16911632
connection::send_batch_inclusion_data_responses(finalized_batch, &batch_merkle_tree).await
16921633
}

0 commit comments

Comments
 (0)