Skip to content

Commit f3be0ea

Browse files
committed
feat: high-availability validator signer
Fixes [A-352](https://linear.app/aztec-labs/issue/A-352/add-postgresql-integration-for-recording-signed-duties) [A-353](https://linear.app/aztec-labs/issue/A-353/implement-core-slash-protection-logic) Introduces High-Availability validator signer. The signer uses a Postgres DB that will be used by multiple sequencer nodes, running with the same set of validator keys, in order to ensure that only one payload per duty is signed.
1 parent d3be5a7 commit f3be0ea

28 files changed

+3604
-4
lines changed

yarn-project/foundation/src/config/env_var.ts

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -313,4 +313,14 @@ export type EnvVar =
313313
| 'FISHERMAN_MODE'
314314
| 'MAX_ALLOWED_ETH_CLIENT_DRIFT_SECONDS'
315315
| 'LEGACY_BLS_CLI'
316-
| 'DEBUG_FORCE_TX_PROOF_VERIFICATION';
316+
| 'DEBUG_FORCE_TX_PROOF_VERIFICATION'
317+
| 'SLASHING_PROTECTION_NODE_ID'
318+
| 'SLASHING_PROTECTION_POLLING_INTERVAL_MS'
319+
| 'SLASHING_PROTECTION_SIGNING_TIMEOUT_MS'
320+
| 'SLASHING_PROTECTION_ENABLED'
321+
| 'VALIDATOR_HA_DATABASE_URL'
322+
| 'VALIDATOR_HA_RUN_MIGRATIONS'
323+
| 'VALIDATOR_HA_POOL_MAX'
324+
| 'VALIDATOR_HA_POOL_MIN'
325+
| 'VALIDATOR_HA_POOL_IDLE_TIMEOUT_MS'
326+
| 'VALIDATOR_HA_POOL_CONNECTION_TIMEOUT_MS';

yarn-project/package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@
6161
"txe",
6262
"test-wallet",
6363
"validator-client",
64+
"validator-ha-signer",
6465
"wallet-sdk",
6566
"world-state"
6667
],

yarn-project/tsconfig.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@
2929
{ "path": "aztec.js/tsconfig.json" },
3030
{ "path": "aztec-node/tsconfig.json" },
3131
{ "path": "validator-client/tsconfig.json" },
32+
{ "path": "validator-ha-signer/tsconfig.json" },
3233
{ "path": "bb-prover/tsconfig.json" },
3334
{ "path": "bot/tsconfig.json" },
3435
{ "path": "constants/tsconfig.json" },
Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
# Database Migrations Guide
2+
3+
This package uses [node-pg-migrate](https://github.com/salsita/node-pg-migrate) for managing database schema changes.
4+
5+
## Quick Reference
6+
7+
```bash
8+
# Run pending migrations
9+
yarn migrate:up
10+
11+
# Rollback last migration
12+
yarn migrate:down
13+
14+
# Check migration status
15+
DATABASE_URL=postgresql://... npx node-pg-migrate status
16+
```
17+
18+
## Migration Files
19+
20+
Migrations are located in the `migrations/` directory and are named with timestamps:
21+
22+
```
23+
migrations/
24+
└── 1_initial-schema.ts
25+
```
26+
27+
## Creating New Migrations
28+
29+
When you need to modify the database schema:
30+
31+
```bash
32+
# Generate a new migration file
33+
npx node-pg-migrate create add-new-field
34+
35+
# This creates: migrations/[timestamp]_add-new-field.ts
36+
```
37+
38+
Edit the generated file:
39+
40+
```typescript
41+
import type { MigrationBuilder } from 'node-pg-migrate';
42+
43+
export async function up(pgm: MigrationBuilder): Promise<void> {
44+
// Add your schema changes here
45+
pgm.addColumn('validator_duties', {
46+
new_field: { type: 'text', notNull: false },
47+
});
48+
}
49+
50+
export async function down(pgm: MigrationBuilder): Promise<void> {
51+
// Reverse the changes
52+
pgm.dropColumn('validator_duties', 'new_field');
53+
}
54+
```
55+
56+
## Production Deployment
57+
58+
### Option 1: Kubernetes Init Container
59+
60+
```yaml
61+
apiVersion: apps/v1
62+
kind: Deployment
63+
metadata:
64+
name: validator
65+
spec:
66+
template:
67+
spec:
68+
initContainers:
69+
- name: db-migrate
70+
image: validator-image:latest
71+
command: ['yarn', 'migrate:up']
72+
env:
73+
- name: DATABASE_URL
74+
valueFrom:
75+
secretKeyRef:
76+
name: db-credentials
77+
key: connection-string
78+
containers:
79+
- name: validator
80+
image: validator-image:latest
81+
# ... validator config
82+
```
83+
84+
### Option 2: Separate Migration Job
85+
86+
```yaml
87+
apiVersion: batch/v1
88+
kind: Job
89+
metadata:
90+
name: validator-migrate-v1
91+
spec:
92+
template:
93+
spec:
94+
containers:
95+
- name: migrate
96+
image: validator-image:latest
97+
command: ['yarn', 'migrate:up']
98+
env:
99+
- name: DATABASE_URL
100+
valueFrom:
101+
secretKeyRef:
102+
name: db-credentials
103+
key: connection-string
104+
restartPolicy: Never
105+
```
106+
107+
### Option 3: CI/CD Pipeline
108+
109+
```yaml
110+
# GitHub Actions example
111+
- name: Run Database Migrations
112+
run: |
113+
export DATABASE_URL=${{ secrets.DATABASE_URL }}
114+
cd yarn-project/validator-ha-signer
115+
yarn migrate:up
116+
```
117+
118+
## High Availability Considerations
119+
120+
The migrations use idempotent SQL operations (`IF NOT EXISTS`, `ON CONFLICT`, etc.), making them safe to run concurrently from multiple nodes. However, for cleaner logs and faster deployments, we recommend:
121+
122+
1. **Run migrations once** from an init container or migration job
123+
2. **Then start** multiple validator nodes
124+
125+
If multiple nodes run migrations simultaneously, they will all succeed, but you'll see redundant log output.
126+
127+
## Development Workflow
128+
129+
```bash
130+
# 1. Create migration
131+
npx node-pg-migrate create my-feature
132+
133+
# 2. Edit migrations/[timestamp]_my-feature.ts
134+
135+
# 3. Test migration locally
136+
export DATABASE_URL=postgresql://localhost:5432/validator_dev
137+
yarn migrate:up
138+
139+
# 4. Test rollback
140+
yarn migrate:down
141+
142+
# 5. Re-apply
143+
yarn migrate:up
144+
145+
# 6. Run tests
146+
yarn test
147+
```
148+
149+
## Troubleshooting
150+
151+
### Migration Failed Midway
152+
153+
If a migration fails partway through:
154+
155+
```bash
156+
# Check the current state
157+
DATABASE_URL=postgresql://... npx node-pg-migrate status
158+
159+
# The failed migration will be marked as running
160+
# Fix the issue and re-run
161+
yarn migrate:up
162+
```
163+
164+
### Reset Development Database
165+
166+
```bash
167+
# Drop all migrations
168+
while yarn migrate:down; do :; done
169+
170+
# Or drop the database entirely
171+
psql -c "DROP DATABASE validator_dev;"
172+
psql -c "CREATE DATABASE validator_dev;"
173+
174+
# Re-run migrations
175+
yarn migrate:up
176+
```
177+
178+
### Check Applied Migrations
179+
180+
```bash
181+
# Query the migrations table
182+
psql $DATABASE_URL -c "SELECT * FROM pgmigrations ORDER BY id;"
183+
```
184+
185+
## Migration Best Practices
186+
187+
1. **Always provide `down()` migrations** for rollback capability
188+
2. **Test migrations on a copy of production data** before deploying
189+
3. **Make migrations backward compatible** when possible
190+
4. **Avoid data migrations in schema migrations** - use separate data migration scripts
191+
5. **Keep migrations small and focused** - one logical change per migration
192+
6. **Never modify committed migrations** - create a new migration instead
Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# Validator HA Signer
2+
3+
Distributed locking and slashing protection for Aztec validators running in high-availability configurations.
4+
5+
## Features
6+
7+
- **Distributed Locking**: Prevents multiple validator nodes from signing the same duty
8+
- **Slashing Protection**: Blocks attempts to sign conflicting data for the same slot
9+
- **Automatic Retry**: Failed signing attempts are cleared, allowing other nodes to retry
10+
- **PostgreSQL Backend**: Shared database for coordination across nodes
11+
12+
## Quick Start
13+
14+
### Option 1: Automatic Migrations (Simplest)
15+
16+
```typescript
17+
import { createHASigner } from '@aztec/validator-ha-signer/factory';
18+
19+
// Migrations run automatically on startup
20+
const { signer, db } = await createHASigner({
21+
databaseUrl: process.env.DATABASE_URL,
22+
enabled: true,
23+
nodeId: 'validator-node-1',
24+
pollingIntervalMs: 100,
25+
signingTimeoutMs: 3000,
26+
runMigrations: true, // Auto-run migrations
27+
});
28+
29+
// Sign with protection
30+
const signature = await signer.signWithProtection(
31+
validatorAddress,
32+
signingRoot,
33+
{ slot: 100n, blockNumber: 50n, dutyType: 'BLOCK_PROPOSAL' },
34+
async root => localSigner.signMessage(root),
35+
);
36+
37+
// Cleanup on shutdown
38+
await db.close();
39+
```
40+
41+
### Option 2: Manual Migrations (Recommended for Production)
42+
43+
```bash
44+
# 1. Run migrations separately (once per deployment)
45+
export DATABASE_URL=postgresql://user:pass@host:port/db
46+
yarn migrate:up
47+
```
48+
49+
```typescript
50+
// 2. Create signer (migrations already applied)
51+
import { createHASigner } from '@aztec/validator-ha-signer/factory';
52+
53+
const { signer, db } = await createHASigner({
54+
databaseUrl: process.env.DATABASE_URL,
55+
enabled: true,
56+
nodeId: 'validator-node-1',
57+
pollingIntervalMs: 100,
58+
signingTimeoutMs: 3000,
59+
// runMigrations defaults to false
60+
});
61+
```
62+
63+
### Advanced: Manual Database Setup
64+
65+
If you need more control over the database connection:
66+
67+
```typescript
68+
import { PostgresSlashingProtectionDatabase, ValidatorHASigner } from '@aztec/validator-ha-signer';
69+
70+
import { Pool } from 'pg';
71+
72+
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
73+
const db = new PostgresSlashingProtectionDatabase(pool);
74+
75+
const signer = new ValidatorHASigner(db, {
76+
enabled: true,
77+
nodeId: 'validator-node-1',
78+
});
79+
```
80+
81+
## Configuration
82+
83+
Set via environment variables or config object:
84+
85+
- `DATABASE_URL`: PostgreSQL connection string (e.g., `postgresql://user:pass@host:port/db`)
86+
- `SLASHING_PROTECTION_NODE_ID`: Unique identifier for this validator node
87+
- `SLASHING_PROTECTION_POLLING_INTERVAL_MS`: How often to check duty status (default: 100)
88+
- `SLASHING_PROTECTION_SIGNING_TIMEOUT_MS`: Max wait for in-progress signing (default: 3000)
89+
90+
## Database Migrations
91+
92+
This package uses `node-pg-migrate` for database schema management.
93+
94+
### Migration Commands
95+
96+
```bash
97+
# Run pending migrations
98+
yarn migrate:up
99+
100+
# Rollback last migration
101+
yarn migrate:down
102+
103+
# Check migration status
104+
DATABASE_URL=postgresql://... npx node-pg-migrate status
105+
```
106+
107+
### Creating New Migrations
108+
109+
```bash
110+
# Generate a new migration file
111+
npx node-pg-migrate create my-migration-name
112+
```
113+
114+
### Production Deployment
115+
116+
Run migrations before starting your application:
117+
118+
```yaml
119+
# Kubernetes example
120+
apiVersion: batch/v1
121+
kind: Job
122+
metadata:
123+
name: validator-db-migrate
124+
spec:
125+
template:
126+
spec:
127+
containers:
128+
- name: migrate
129+
image: your-validator-image
130+
command: ['yarn', 'migrate:up']
131+
env:
132+
- name: DATABASE_URL
133+
valueFrom:
134+
secretKeyRef:
135+
name: db-secret
136+
key: url
137+
restartPolicy: OnFailure
138+
```
139+
140+
## How It Works
141+
142+
When multiple validator nodes attempt to sign:
143+
144+
1. First node acquires lock and signs
145+
2. Other nodes receive `DutyAlreadySignedError` (expected)
146+
3. If different data detected: `SlashingProtectionError` (critical)
147+
4. Failed attempts are auto-cleaned, allowing retry
148+
149+
## Development
150+
151+
```bash
152+
yarn build # Build package
153+
yarn test # Run tests
154+
yarn clean # Clean build artifacts
155+
```
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
import config from '@aztec/foundation/eslint';
2+
3+
export default config;

0 commit comments

Comments
 (0)