Minifed vs unminified bundle benchmarks

[cb1kenobi]

Run ./compare-bundles.sh on Mac/Linux, ./compare-bundles.cmd on Windows.

It will build a minified bundle, then benchmark it 50 times. Afterwards, it does a unminified bundle and runs the benchmark 50 times.

Based on my Mac M4 Pro, the minified bundle is consistently 0.248 to 0.29 ms faster than the unminified bundle, which makes logical sense.

On Windows, I'm getting mixed results. Sometimes the unminified bundle is 0.168 to 3.167 ms faster than the minified bundle. Other times the minified bundle is 0.115 to 0.903 ms faster than the unminified bundle.

On WSL, I'm also getting mixed results. Sometimes the unminified is .249 to 1.935 ms faster. Other times the minified bundle is 0.006 ms faster.

Note: This PR is not meant to be merged.

[github-actions]

📊 Benchmark Results

get-sync.bench.ts

getSync() > random keys - small key size (100 records)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	21.62K ops/sec	46.25	41.46	420.47	0.621	10,812
🥈 rocksdb	2	4.48K ops/sec	223.444	206.861	2,946.88	1.30	2,238

getSync() > sequential keys - small key size (100 records)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	24.54K ops/sec	40.75	35.87	4,478.924	1.96	12,271
🥈 rocksdb	2	4.86K ops/sec	205.615	196.316	688.309	0.355	2,432

ranges.bench.ts

getRange() > small range (100 records, 50 range)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	23.74K ops/sec	42.13	36.97	492.083	0.992	11,868
🥈 rocksdb	2	3.47K ops/sec	287.976	250.364	2,696.328	1.55	1,737

realistic-load.bench.ts

Realistic write load with workers > write variable records with transaction log

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	175.80 ops/sec	5,688.349	74.35	156,635.714	47.69	352
🥈 lmdb	2	25.22 ops/sec	39,648.4	283.464	1,246,424.647	137.301	64.00

transaction-log.bench.ts

Transaction log > read 100 iterators while write log with 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	33.25K ops/sec	30.07	17.79	500.81	0.697	16,626
🥈 lmdb	2	411.92 ops/sec	2,427.669	164.622	10,415.72	5.58	206

Transaction log > read one entry from random position from log with 1000 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	582.69K ops/sec	1.72	1.49	428.032	0.394	291,345
🥈 lmdb	2	286.55K ops/sec	3.49	1.79	5,184.602	4.58	143,276

worker-put-sync.bench.ts

putSync() > random keys - small key size (100 records, 10 workers)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	878.29 ops/sec	1,138.572	994.842	1,807.673	0.300	1,757
🥈 lmdb	2	0.94 ops/sec	1,064,851.613	938,209.419	1,106,572.434	3.42	10.00

worker-transaction-log.bench.ts

Transaction log with workers > write log with 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	18.06K ops/sec	55.36	30.79	6,113.071	0.839	36,130
🥈 lmdb	2	776.34 ops/sec	1,288.097	129.587	13,192.775	5.89	1,554

---

Results from commit ff7cda4

[kriszyp]

This is what I got with a couple runs:

kzyp@kzyp-XPS-15-9520:~/dev/rocksdb-js$ ./compare-bundles.sh 
================================
Bundle Load Performance Comparison
Iterations: 50
================================

Building minified bundle...
Minified size: 17.63 KB

Benchmarking minified bundle...
Bundle Load Performance Benchmark (Fast)

Bundle size: 17.63 KB

Measuring first-load performance (50 fresh processes)...

Progress: 50/50

=== First Load Results ===
  Mean:   22.329 ms
  Median: 21.929 ms
  Min:    17.857 ms
  Max:    33.782 ms
  StdDev: 3.300 ms
  P95:    29.834 ms
  P99:    33.782 ms

Building unminified bundle...
Unminified size: 50.55 KB

Benchmarking unminified bundle...
Bundle Load Performance Benchmark (Fast)

Bundle size: 50.55 KB

Measuring first-load performance (50 fresh processes)...

Progress: 50/50

=== First Load Results ===
  Mean:   21.959 ms
  Median: 21.256 ms
  Min:    16.925 ms
  Max:    33.491 ms
  StdDev: 3.550 ms
  P95:    29.244 ms
  P99:    33.491 ms

================================
COMPARISON
================================
Size difference: 32.92 KB (186.7% larger)

✗ Unminified is faster by .673 ms (3.1% improvement)

This suggests V8 parse/compile time dominates over network transfer.

Results saved in /tmp/minified-results.txt and /tmp/unminified-results.txt
Restored minified bundle to dist/index.mjs
kzyp@kzyp-XPS-15-9520:~/dev/rocksdb-js$ ./compare-bundles.sh 
================================
Bundle Load Performance Comparison
Iterations: 50
================================

Building minified bundle...
Minified size: 17.63 KB

Benchmarking minified bundle...
Bundle Load Performance Benchmark (Fast)

Bundle size: 17.63 KB

Measuring first-load performance (50 fresh processes)...

Progress: 50/50

=== First Load Results ===
  Mean:   19.743 ms
  Median: 19.949 ms
  Min:    16.425 ms
  Max:    30.270 ms
  StdDev: 2.666 ms
  P95:    24.905 ms
  P99:    30.270 ms

Building unminified bundle...
Unminified size: 50.55 KB

Benchmarking unminified bundle...
Bundle Load Performance Benchmark (Fast)

Bundle size: 50.55 KB

Measuring first-load performance (50 fresh processes)...

Progress: 50/50

=== First Load Results ===
  Mean:   19.108 ms
  Median: 18.215 ms
  Min:    16.428 ms
  Max:    33.168 ms
  StdDev: 2.803 ms
  P95:    23.030 ms
  P99:    33.168 ms

================================
COMPARISON
================================
Size difference: 32.92 KB (186.7% larger)

✗ Unminified is faster by 1.734 ms (9.5% improvement)

This suggests V8 parse/compile time dominates over network transfer.

Results saved in /tmp/minified-results.txt and /tmp/unminified-results.txt
Restored minified bundle to dist/index.mjs

Unminified was actually faster on my machine.

Anyway, this certainly is not urgent or that important. Just thought the difference might be small enough that it might be preferable to keep the more readable builds, but I certainly know how to use unminified for my own local development.

[kriszyp]

consistently 0.29 ms to 0.248 ms than

I realized I actually don't what difference you saw, I think this is missing the "greater" or "lesser" :)