Skip to content

Commit 48a234c

Browse files
committed
Update benchmark numbers from latest 10M run, prune invalidated 100M results
Incorporate new4 benchmark output (commit a094df7): honest 1T window numbers, corrected CB-CD-GRP, improved LIKE/string/CORR ratios. Remove 12 invalidated 100M rows (WIN, LIKE, H2O-Q6/Q8/Q9, DS-Q98, Q27). 35 of 46 wins at 10M, 15 of 34 at 100M.
1 parent a094df7 commit 48a234c

File tree

2 files changed

+112
-131
lines changed

2 files changed

+112
-131
lines changed

README.md

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -49,38 +49,38 @@ Stratum vs DuckDB (10M rows, single-threaded, best of array/index mode)
4949
══════════════════════════════════════════════════════════════════════════
5050
Stratum DuckDB Ratio
5151
──────────────────────────────────────────────────────────────────────
52-
TPC-H Q6 (filter+sum-product) 15ms 31ms 2.1x faster
53-
SSB Q1.1 (filter+sum-product) 14ms 30ms 2.1x faster
54-
Filtered COUNT (NEQ pred) 4ms 13ms 3.6x faster
55-
Group-by COUNT (3 groups) 19ms 27ms 1.4x faster
56-
TPC-H Q1 (7 aggs, 4 groups) 79ms 98ms 1.2x faster
57-
H2O Q3 (100K string groups) 70ms 380ms 5.4x faster
58-
H2O Q2 (10K string groups) 32ms 126ms 3.9x faster
59-
H2O Q6 (STDDEV, 10K groups) 31ms 83ms 2.7x faster
60-
H2O Q9 (CORR, 10K groups) 68ms 136ms 2.0x faster
61-
H2O Q10 (10M groups, 6 cols) 842ms 7294ms 8.7x faster
62-
LIKE '%search%' (string scan) 54ms 252ms 4.7x faster
63-
AVG(LENGTH(URL)) (string fn) 40ms 179ms 4.5x faster
52+
TPC-H Q6 (filter+sum-product) 13ms 28ms 2.2x faster
53+
SSB Q1.1 (filter+sum-product) 13ms 28ms 2.2x faster
54+
Filtered COUNT (NEQ pred) 3ms 12ms 4.0x faster
55+
Group-by COUNT (3 groups) 17ms 24ms 1.4x faster
56+
TPC-H Q1 (7 aggs, 4 groups) 75ms 93ms 1.2x faster
57+
H2O Q3 (100K string groups) 71ms 362ms 5.1x faster
58+
H2O Q2 (10K string groups) 33ms 123ms 3.7x faster
59+
H2O Q6 (STDDEV, 10K groups) 30ms 81ms 2.7x faster
60+
H2O Q9 (CORR, 10K groups) 61ms 134ms 2.2x faster
61+
H2O Q10 (10M groups, 6 cols) 832ms 7056ms 8.5x faster
62+
LIKE '%search%' (string scan) 47ms 240ms 5.1x faster
63+
AVG(LENGTH(URL)) (string fn) 38ms 170ms 4.5x faster
6464
──────────────────────────────────────────────────────────────────────
65-
COUNT(DISTINCT) 1.1M 116ms 62ms DuckDB 1.9x
66-
COUNT WHERE sparse pred 21ms 5ms DuckDB 4.2x
65+
COUNT(DISTINCT) 1.1M 109ms 61ms DuckDB 1.8x
66+
COUNT WHERE sparse pred 22ms 5ms DuckDB 4.9x
6767
──────────────────────────────────────────────────────────────────────
6868
```
6969

7070
**Summary (10M rows, single-threaded, best of array/index mode vs DuckDB 1T):**
7171

7272
| Category | Queries | Stratum faster | DuckDB faster |
7373
|----------|---------|---------------|--------------|
74-
| TPC-H / SSB | 7 | 7 (1.2–3.6x) | 0 |
75-
| H2O Group-By | 10 | 10 (1.0–8.7x) | 0 |
74+
| TPC-H / SSB | 7 | 7 (1.2–4.0x) | 0 |
75+
| H2O Group-By | 10 | 9 (1.0–8.5x) | 1 |
7676
| H2O Join | 3 | 2 (1.1x) | 1 |
77-
| ClickBench | 22 | 16 (1.0–119x) | 6 |
78-
| NYC Taxi | 4 | 2 (1.1–1.4x) | 2 |
77+
| ClickBench | 22 | 15 (1.0–105x) | 7 |
78+
| NYC Taxi | 4 | 2 (1.0–1.4x) | 2 |
7979
| Statistical | 5 | 5 (1.1–2.4x) | 0 |
80-
| Window | 3 | 3 (2.4–3.2x) | 0 |
81-
| Hash Join + TPC-DS | 3 | 2 (1.9–2.9x) | 1 |
80+
| Window | 3 | 3 (1.3–2.1x) | 0 |
81+
| Hash Join + TPC-DS | 3 | 2 (1.3–2.1x) | 1 |
8282

83-
DuckDB wins on sparse-selectivity filters and global COUNT(DISTINCT). Stratum wins on arithmetic-heavy aggregates, string operations, group-by, LIKE, statistical aggregates, and window functions.
83+
DuckDB wins on sparse-selectivity filters, global COUNT(DISTINCT), and window-based top-N. Stratum wins on arithmetic-heavy aggregates, string operations, group-by, LIKE, statistical aggregates, and window functions.
8484

8585
Run benchmarks yourself:
8686

0 commit comments

Comments
 (0)