feat: add inkwell crate

Author: evansims

CLAUDE.md

@@ -75,7 +75,7 @@ cargo run -p ledger-server --release -- --config config.toml
 │   Raft via openraft │ Log storage │ Batching │ Idempotency  │
 ├─────────────────────────────────────────────────────────────┤
 │                 ledger-storage (state)                      │
-│   redb engine │ Entity/Relationship stores │ Indexes        │
+│   inkwell B+ tree │ Entity/Relationship stores │ Indexes    │
 ├─────────────────────────────────────────────────────────────┤
 │                 ledger-types (shared)                       │
 │   Hash primitives │ Merkle proofs │ Config │ Error types    │
@@ -85,13 +85,13 @@ cargo run -p ledger-server --release -- --config config.toml
 **Crates:**
 
 - `ledger-types` — Core types, SHA-256/seahash, merkle tree, snafu errors
-- `ledger-storage` — redb wrapper, entity/relationship CRUD, dual indexes, state root computation
+- `ledger-storage` — inkwell wrapper, entity/relationship CRUD, dual indexes, state root computation
 - `ledger-raft` — openraft integration, log storage, gRPC services, transaction batching
 - `ledger-server` — Main binary, bootstrap, config loading
 
 **Key abstractions:**
 
-- `StorageEngine` (storage/engine.rs) — redb database wrapper with transaction helpers
+- `StorageEngine` (storage/engine.rs) — inkwell database wrapper with transaction helpers
 - `StateLayer` (storage/state.rs) — Applies blocks, computes bucket-based state roots
 - `LedgerServer` (raft/server.rs) — gRPC server combining all services with Raft
 

Cargo.lock

@@ -5,6 +5,7 @@ members = [
     "crates/storage",
     "crates/raft",
     "crates/server",
+    "crates/inkwell",
 ]
 
 [workspace.package]
@@ -29,9 +30,6 @@ tower = { version = "0.5", features = ["limit", "load-shed", "timeout"] }
 # Raft consensus
 openraft = { version = "0.9", features = ["serde", "tracing-log"] }
 
-# Storage
-redb = "2.4"
-
 # Crypto
 sha2 = "0.10"
 seahash = "4.1"
@@ -106,6 +104,12 @@ async-stream = "0.3"
 ledger-types = { path = "crates/types" }
 ledger-storage = { path = "crates/storage" }
 ledger-raft = { path = "crates/raft" }
+inkwell = { path = "crates/inkwell" }
+
+# Hashing (for inkwell)
+xxhash-rust = { version = "0.8", features = ["xxh32", "xxh3"] }
+byteorder = "1.5"
+memmap2 = "0.9"
 
 [workspace.lints.rust]
 unsafe_code = "deny"

Cargo.toml

@@ -81,7 +81,7 @@ org namespace (per-organization)
 - [Durability & Finality Model](#durability--finality-model)
 - [Persistent Storage Architecture](#persistent-storage-architecture)
   - [Directory Layout](#directory-layout)
-  - [Storage Backend: redb](#storage-backend-redb)
+  - [Storage Backend: inkwell](#storage-backend-inkwell)
   - [Block Archive Format](#block-archive-format)
   - [Snapshot Format](#snapshot-format)
   - [Crash Recovery](#crash-recovery)
@@ -394,7 +394,7 @@ graph TD
 
 **What's NOT merkleized per-write:**
 
-- Individual index entries (stored in redb for O(1) lookup)
+- Individual index entries (stored in inkwell for O(1) lookup)
 - Dual indexes for relationship traversal
 
 **Verification approach:**
@@ -405,7 +405,7 @@ graph TD
 
 ```rust
 struct StateLayer {
-    // Fast K/V storage for queries (redb)
+    // Fast K/V storage for queries (inkwell)
     kv: Database,
 
     // Indexes for relationship traversal
@@ -550,7 +550,7 @@ fn decode_storage_key(storage_key: &[u8]) -> (VaultId, u8, &[u8]) {
 
 #### Per-Vault Bucket Structure
 
-Each vault maintains independent bucket tracking. Vaults in the same shard share a redb instance but compute separate `state_root` values.
+Each vault maintains independent bucket tracking. Vaults in the same shard share an inkwell database but compute separate `state_root` values.
 
 ```rust
 struct ShardState {
@@ -655,7 +655,7 @@ impl ShardState {
 State root computation must be identical on all nodes:
 
 1. **Hash function**: seahash for bucket assignment, SHA-256 for roots
-2. **Iteration order**: redb guarantees lexicographic key order within range
+2. **Iteration order**: inkwell guarantees lexicographic key order within range
 3. **Key-value encoding**: Length-prefixed (prevents ambiguous concatenation)
 4. **Empty bucket**: Hash of empty input → `SHA-256("")`
 
@@ -1590,7 +1590,7 @@ Scaling dimensions:
 
 | Metric             | Target          | Measurement                    | Rationale                         |
 | ------------------ | --------------- | ------------------------------ | --------------------------------- |
-| Read (p50)         | <0.5ms          | Single key, no proof, follower | redb lookup + gRPC                |
+| Read (p50)         | <0.5ms          | Single key, no proof, follower | inkwell lookup + gRPC             |
 | Read (p99)         | <2ms            | Single key, no proof, follower | Tail latency from GC/compaction   |
 | Read + proof (p99) | <10ms           | With merkle proof generation   | Bucket-based O(k) proof           |
 | Write (p50)        | <10ms           | Single tx, quorum commit       | Raft RTT + fsync                  |
@@ -1600,7 +1600,7 @@ Scaling dimensions:
 
 **Why these targets are achievable:**
 
-- **Read p99 <2ms**: Follower reads bypass Raft consensus. redb B-tree lookup is O(log n). etcd achieves ~2ms p99 for serializable reads.
+- **Read p99 <2ms**: Follower reads bypass Raft consensus. inkwell B+ tree lookup is O(log n). etcd achieves ~2ms p99 for serializable reads.
 - **Write p99 <50ms**: Aggressive for blockchain but achievable because:
   - Bucket-based state root: O(k) where k = dirty keys, not O(n) full tree
   - Single Raft RTT: ~1-2ms same datacenter
@@ -2583,17 +2583,17 @@ Each node uses a single data directory with subdirectories per concern:
 ├── shards/
 │   ├── _system/                 # System Raft group
 │   │   ├── raft/                # Openraft storage
-│   │   │   ├── log.redb         # Raft log entries
+│   │   │   ├── log.inkwell         # Raft log entries
 │   │   │   └── vote             # Current term + voted_for
-│   │   ├── state.redb           # State machine (system registry)
+│   │   ├── state.inkwell           # State machine (system registry)
 │   │   └── snapshots/
 │   │       ├── 000001000.snap   # Snapshot at height 1000
 │   │       └── 000002000.snap   # Snapshot at height 2000
 │   ├── shard_0001/              # User shard group
 │   │   ├── raft/
-│   │   │   ├── log.redb
+│   │   │   ├── log.inkwell
 │   │   │   └── vote
-│   │   ├── state.redb           # State for all vaults in shard
+│   │   ├── state.inkwell           # State for all vaults in shard
 │   │   ├── blocks/
 │   │   │   ├── segment_000000.blk   # Blocks 0-9999
 │   │   │   ├── segment_000001.blk   # Blocks 10000-19999
@@ -2610,26 +2610,26 @@ Each node uses a single data directory with subdirectories per concern:
 | Decision                      | Rationale                                                  |
 | ----------------------------- | ---------------------------------------------------------- |
 | Per-shard directories         | Matches Raft group boundaries; independent failure domains |
-| Separate raft/ and state.redb | Raft log is append-heavy; state is random-access heavy     |
+| Separate raft/ and state.inkwell | Raft log is append-heavy; state is random-access heavy     |
 | Block segments                | Append-only writes; easy archival of old segments          |
 | Snapshots by height           | Predictable naming; simple retention policy                |
 
-### Storage Backend: redb
+### Storage Backend: inkwell
 
-[redb](https://github.com/cberner/redb) provides ACID transactions with a simple API. Each shard uses two redb databases:
+inkwell is our custom B+ tree storage engine providing ACID transactions with MVCC. Each shard uses two inkwell databases:
 
-**raft/log.redb** — Raft log storage:
+**raft/log.inkwell** — Raft log storage:
 
 ```rust
-// Tables in log.redb
+// Tables in log.inkwell
 const LOG_ENTRIES: TableDefinition<u64, &[u8]> = TableDefinition::new("log");
 const LOG_META: TableDefinition<&str, &[u8]> = TableDefinition::new("meta");
 
-struct RedbLogStorage {
+struct InkwellLogStorage {
     db: Database,
 }
 
-impl RaftLogStorage<TypeConfig> for RedbLogStorage {
+impl RaftLogStorage<TypeConfig> for InkwellLogStorage {
     async fn get_log_state(&mut self) -> Result<LogState<TypeConfig>> {
         let txn = self.db.begin_read()?;
         let table = txn.open_table(LOG_META)?;
@@ -2712,10 +2712,10 @@ impl RaftLogStorage<TypeConfig> for RedbLogStorage {
 }
 ```
 
-**state.redb** — State machine storage:
+**state.inkwell** — State machine storage:
 
 ```rust
-// Tables in state.redb (per shard, contains all vaults)
+// B+ tree tables in state.inkwell (per shard, contains all vaults)
 const RELATIONSHIPS: TableDefinition<&[u8], &[u8]> = TableDefinition::new("rel");
 const ENTITIES: TableDefinition<&[u8], &[u8]> = TableDefinition::new("ent");
 const OBJ_INDEX: TableDefinition<&[u8], &[u8]> = TableDefinition::new("obj_idx");
@@ -2730,7 +2730,7 @@ fn make_key(vault_id: VaultId, key: &[u8]) -> Vec<u8> {
 }
 ```
 
-**Per-shard isolation**: All vaults in a shard share a single redb instance. Keys are prefixed with `vault_id` for isolation. This reduces file handle count and enables cross-vault operations within a shard if needed.
+**Per-shard isolation**: All vaults in a shard share a single inkwell database. Keys are prefixed with `vault_id` for isolation. This reduces file handle count and enables cross-vault operations within a shard if needed.
 
 ### Block Archive Format
 
@@ -2886,7 +2886,7 @@ impl Node {
             let shard_id = parse_shard_id(&shard_dir.file_name())?;
 
             // 3. Recover Raft state
-            let raft_storage = RedbLogStorage::open(shard_dir.join("raft/log.redb"))?;
+            let raft_storage = RedbLogStorage::open(shard_dir.join("raft/log.inkwell"))?;
             let vote = raft_storage.read_vote().await?;
             let log_state = raft_storage.get_log_state().await?;
 
@@ -2936,7 +2936,7 @@ impl Node {
 | Failure Mode         | Recovery Action                               |
 | -------------------- | --------------------------------------------- |
 | Clean shutdown       | Replay from last snapshot + committed log     |
-| Crash during write   | Incomplete redb txn rolled back automatically |
+| Crash during write   | Incomplete inkwell txn rolled back automatically |
 | Corrupted snapshot   | Skip to older snapshot, replay more log       |
 | Corrupted log entry  | Fetch from peer, or rebuild from snapshot     |
 | Missing segment file | Fetch from peer (block archive is replicated) |
@@ -2962,7 +2962,7 @@ impl Node {
 ### Storage Invariants
 
 30. **Raft log durability**: Log entries are fsync'd before Raft acknowledgment
-31. **State consistency**: `state.redb` reflects all applied log entries up to `applied_index`
+31. **State consistency**: `state.inkwell` reflects all applied log entries up to `applied_index`
 32. **Block archive append-only**: Segment files are never modified after creation (only new segments appended)
 33. **Snapshot validity**: Snapshot `state_root` matches block header at `shard_height`
 

DESIGN.md

@@ -60,7 +60,7 @@ Ledger is InferaDB's persistence layer — a blockchain database for authorizati
 │  ├─ Vault: prod [chain]       │  ├─ Vault: main [chain]     │
 │  └─ Vault: staging [chain]    │  └─ ...                     │
 ├─────────────────────────────────────────────────────────────┤
-│                   State Layer (redb)                        │
+│                  State Layer (inkwell)                      │
 │    Relationships │ Entities │ Indexes │ State Roots         │
 └─────────────────────────────────────────────────────────────┘
 ```

README.md

@@ -0,0 +1,37 @@
+[package]
+name = "inkwell"
+version.workspace = true
+edition = "2021"
+description = "Purpose-built embedded storage engine for InferaDB Ledger"
+license.workspace = true
+repository.workspace = true
+authors.workspace = true
+
+[dependencies]
+# Hashing for checksums
+xxhash-rust = { workspace = true }
+
+# Error handling
+thiserror = { workspace = true }
+
+# Serialization for key/value encoding
+byteorder = { workspace = true }
+
+# Memory-mapped files
+memmap2 = { workspace = true }
+
+# Parking lot for faster locks
+parking_lot = { workspace = true }
+
+# Logging
+tracing = { workspace = true }
+
+[dev-dependencies]
+tempfile = { workspace = true }
+proptest = { workspace = true }
+rand = { workspace = true }
+
+[features]
+default = []
+# Enable additional safety checks (slower but catches bugs)
+paranoid = []