|
| 1 | +# GroveDB Benchmark Implementation Plan |
| 2 | + |
| 3 | +## Context |
| 4 | + |
| 5 | +All 7 phases (28 commits) of the GroveDB FFI are complete on `grovedb_p2`. Benchmarks |
| 6 | +are needed for each API phase (0-6) so they can be interleaved between the implementation |
| 7 | +commits. They use **nanobench** (vendored at `src/bench/vendor/nanobench.h` v4.3.11) and |
| 8 | +follow Dash Core's `BenchRunner` + `BENCHMARK()` macro auto-registration pattern. |
| 9 | + |
| 10 | +GroveDB-specific `OperationCost` fields appear **inside the markdown table** alongside |
| 11 | +wall-clock timing, since these are used for fee calculation in Dash Platform. |
| 12 | + |
| 13 | +--- |
| 14 | + |
| 15 | +## Coding Rules |
| 16 | + |
| 17 | +- **Indent**: 2 spaces for `.h`, `.cpp` |
| 18 | +- **Namespaces**: Nested definition (`namespace grovedb {`), never `namespace xx::yy` |
| 19 | +- **Header guards**: `GROVEDB_` prefix for `src/` headers |
| 20 | +- **Header visibility**: Bench targets only use public headers from `include/grovedb/` |
| 21 | +- **Exception averse**: Never throw our own exceptions |
| 22 | +- **Build**: Bench is **Meson-only** (no `sources.mk` entries) |
| 23 | + |
| 24 | +--- |
| 25 | + |
| 26 | +## Key Design Decisions |
| 27 | + |
| 28 | +### 1. OperationCost IN the Markdown Table |
| 29 | + |
| 30 | +1. `bench.output(nullptr)` - suppress nanobench's default table |
| 31 | +2. Before each `bench.run()`, pre-execute the operation once for `OperationCost`, |
| 32 | + store as context via `SetCostContext(bench, cost)` |
| 33 | +3. After all benchmarks, call `RenderWithCosts(results, std::cout)` - iterates |
| 34 | + results and formats timing + cost columns into a markdown table |
| 35 | + |
| 36 | +### 2. Framework: Dash Core BenchRunner Pattern |
| 37 | + |
| 38 | +- `BenchFunction = std::function<void(ankerl::nanobench::Bench&)>` |
| 39 | +- `BenchRunner` with static `benchmarks()` map, constructor-based auto-registration |
| 40 | +- `BENCHMARK(n)` macro creating a static `BenchRunner` at file scope |
| 41 | +- `Args` struct with `is_list_only` and `regex_filter` fields |
| 42 | + |
| 43 | +### 3. Shared Infrastructure |
| 44 | + |
| 45 | +- `TempDir` from `src/test/util/` |
| 46 | +- `MakeKey(i, len)`, `MakeValue(i, len)` - deterministic byte generation |
| 47 | +- `PopulateDb(db, n, key_len, val_len)` - pre-fill a tree |
| 48 | +- `SetCostContext(bench, cost)` / `ClearCostContext(bench)` - cost context |
| 49 | + |
| 50 | +--- |
| 51 | + |
| 52 | +## Benchmark Catalog |
| 53 | + |
| 54 | +### Phase 0: Element (`bench_element.cpp`) |
| 55 | + |
| 56 | +| Benchmark | API | |
| 57 | +|-----------|-----| |
| 58 | +| `ElementItem` | `Element::Item(bytes)` | |
| 59 | +| `ElementEmptyTree` | `Element::EmptyTree()` | |
| 60 | +| `ElementEmptySumTree` | `Element::EmptySumTree()` | |
| 61 | +| `ElementSumItem` | `Element::SumItem(42)` | |
| 62 | + |
| 63 | +### Phase 1: CRUD (`bench_crud.cpp`) |
| 64 | + |
| 65 | +| Benchmark | API | |
| 66 | +|-----------|-----| |
| 67 | +| `Open` | `Db::Open` | |
| 68 | +| `Flush` | `Db::Flush` | |
| 69 | +| `GetRootHash` | `Db::GetRootHash` | |
| 70 | +| `PutSequential` | `Db::Put` (sequential keys) | |
| 71 | +| `PutRandom` | `Db::Put` (random keys into 10K tree) | |
| 72 | +| `GetHit` | `Db::Get` (known key) | |
| 73 | +| `GetMiss` | `Db::Get` (non-existent key) | |
| 74 | +| `GetDirect` | `Db::GetDirect` | |
| 75 | +| `GetOptionalHit` | `Db::GetOptional` (existing key) | |
| 76 | +| `GetOptionalMiss` | `Db::GetOptional` (missing key) | |
| 77 | +| `KeyExistsHit` | `Db::KeyExists` (existing key) | |
| 78 | +| `KeyExistsMiss` | `Db::KeyExists` (missing key) | |
| 79 | +| `SubtreeExists` | `Db::SubtreeExists` (3-level path) | |
| 80 | +| `IsEmptyTree` | `Db::IsEmptyTree` | |
| 81 | +| `PutIfAbsentNew` | `Db::PutIfAbsent` (key absent) | |
| 82 | +| `PutIfAbsentExists` | `Db::PutIfAbsent` (key present) | |
| 83 | +| `PutIfChangedSame` | `Db::PutIfChanged` (same value) | |
| 84 | +| `PutIfChangedDiff` | `Db::PutIfChanged` (different value) | |
| 85 | +| `TxnBeginCommitEmpty` | `BeginTransaction` + `Commit` | |
| 86 | +| `TxnPutCommit` | `Put` in txn + `Commit` | |
| 87 | +| `TxnGetOverhead` | `Get` with txn vs plain | |
| 88 | + |
| 89 | +### Phase 2: Delete (`bench_delete.cpp`) |
| 90 | + |
| 91 | +| Benchmark | API | |
| 92 | +|-----------|-----| |
| 93 | +| `Delete` | `Db::Delete` | |
| 94 | +| `DeleteIfEmptyTrue` | `Db::DeleteIfEmpty` (empty subtree) | |
| 95 | +| `DeleteIfEmptyFalse` | `Db::DeleteIfEmpty` (non-empty) | |
| 96 | +| `PruneEmptyAncestors` | `Db::PruneEmptyAncestors` | |
| 97 | +| `Clear` | `Db::Clear` | |
| 98 | + |
| 99 | +### Phase 3: Query (`bench_query.cpp`) |
| 100 | + |
| 101 | +| Benchmark | API | |
| 102 | +|-----------|-----| |
| 103 | +| `QueryValuesKey` | `QueryValues` + `QueryItem::Key` | |
| 104 | +| `QueryValuesRange10` | `QueryValues` + `RangeFull` limit=100 | |
| 105 | +| `QueryValuesRange100` | `QueryValues` + `RangeFull` (full scan) | |
| 106 | +| `QueryValuesLimit` | `QueryValues` limit=10 on 10K tree | |
| 107 | +| `QueryRawElement` | `QueryRaw` result_type=0 | |
| 108 | +| `QueryRawKeyElement` | `QueryRaw` result_type=1 | |
| 109 | +| `QueryRawPathKeyElement` | `QueryRaw` result_type=2 | |
| 110 | +| `QuerySums` | `QuerySums` (1K SumItems) | |
| 111 | +| `QueryItemsOrSums` | `QueryItemsOrSums` | |
| 112 | +| `QueryKeysOptional` | `QueryKeysOptional` | |
| 113 | + |
| 114 | +### Phase 4: Proof (`bench_proof.cpp`) |
| 115 | + |
| 116 | +| Benchmark | API | |
| 117 | +|-----------|-----| |
| 118 | +| `ProveSingleKey` | `Db::Prove` (1 Key item) | |
| 119 | +| `ProveRange` | `Db::Prove` (Range item) | |
| 120 | +| `VerifyQuery` | `Db::VerifyQuery` | |
| 121 | +| `VerifySubsetQuery` | `Db::VerifySubsetQuery` | |
| 122 | +| `ProveVerifyRoundTrip` | `Prove` + `VerifyQuery` | |
| 123 | + |
| 124 | +### Phase 5: Batch (`bench_batch.cpp`) |
| 125 | + |
| 126 | +| Benchmark | API | |
| 127 | +|-----------|-----| |
| 128 | +| `BatchInsert10` | `ApplyBatch` (10 ops) | |
| 129 | +| `BatchInsert100` | `ApplyBatch` (100 ops) | |
| 130 | +| `BatchInsert1000` | `ApplyBatch` (1000 ops) | |
| 131 | +| `BatchMixed100` | `ApplyBatch` (50 insert + 50 delete) | |
| 132 | +| `BatchReplace100` | `ApplyBatch` (100 Replace ops) | |
| 133 | +| `BatchDeleteTree` | `ApplyBatch` (subtree deletion) | |
| 134 | +| `BatchInTxn100` | `ApplyBatch(ops, opts, txn)` | |
| 135 | + |
| 136 | +### Phase 6: Auxiliary + Checkpoint (`bench_aux.cpp`) |
| 137 | + |
| 138 | +| Benchmark | API | |
| 139 | +|-----------|-----| |
| 140 | +| `PutAux` | `Db::PutAux` | |
| 141 | +| `GetAuxHit` | `Db::GetAux` (existing key) | |
| 142 | +| `GetAuxMiss` | `Db::GetAux` (missing key) | |
| 143 | +| `DeleteAux` | `Db::DeleteAux` | |
| 144 | +| `FindSubtrees` | `Db::FindSubtrees` | |
| 145 | +| `CreateCheckpoint` | `Db::CreateCheckpoint` | |
| 146 | + |
| 147 | +--- |
| 148 | + |
| 149 | +## Verification |
| 150 | + |
| 151 | +```bash |
| 152 | +# Setup bench build directory |
| 153 | +meson setup /tmp/grovedb_benchdir src/ffi/grovedb \ |
| 154 | + -Dbuild_bench=true \ |
| 155 | + -Dgrovedb_cxx_build_dir=$(pwd)/rust/builddir/subprojects/grovedb_cxx |
| 156 | + |
| 157 | +# Compile |
| 158 | +meson compile -C /tmp/grovedb_benchdir |
| 159 | + |
| 160 | +# Run all benchmarks |
| 161 | +/tmp/grovedb_benchdir/src/bench/bench_grovedb |
| 162 | + |
| 163 | +# Run filtered |
| 164 | +/tmp/grovedb_benchdir/src/bench/bench_grovedb --filter="Get.*" |
| 165 | + |
| 166 | +# List only |
| 167 | +/tmp/grovedb_benchdir/src/bench/bench_grovedb --list |
| 168 | +``` |
0 commit comments