fix realtime #14

jaredLunde · 2025-08-07T03:09:13Z

No description provided.

github-actions · 2025-08-07T03:10:09Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │              new.bench               │
                                                       │    sec/op    │    sec/op     vs base                │
Write_SingleEntry-4                                      808.3n ± ∞ ¹   885.1n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       1.981µ ± ∞ ¹   2.035µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       21.79µ ± ∞ ¹   24.29µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                        315.2µ ± ∞ ¹   315.8µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                        3.037m ± ∞ ¹   2.691m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              81.52µ ± ∞ ¹   80.66µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.960µ ± ∞ ¹   1.515µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.17µ ± ∞ ¹   12.11µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      15.80µ ± ∞ ¹   17.86µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    116.4µ ± ∞ ¹   114.7µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     194.2µ ± ∞ ¹   166.7µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.201m ± ∞ ¹   1.202m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4                                       943.7n ± ∞ ¹   991.6n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.004µ ± ∞ ¹   1.082µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.562µ ± ∞ ¹   1.593µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentReadWrite-4                              3.027µ ± ∞ ¹   3.063µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_SequentialRead-4                                143.7µ ± ∞ ¹   143.6µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_RandomAccess-4                                  138.0µ ± ∞ ¹   139.9µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Consumer_LagMeasurement-4                                255.8n ± ∞ ¹   237.6n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.064µ ± ∞ ¹   2.649µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  16.06µ         16.17µ        +0.68%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            14.98 ± ∞ ¹   13.36 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_SmallBatch-4             47.01 ± ∞ ¹   45.46 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4             61.83 ± ∞ ¹   61.47 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              50.04 ± ∞ ¹   65.64 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4              68.22 ± ∞ ¹   58.63 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4             9.350 ± ∞ ¹   9.034 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.636 ± ∞ ¹   8.946 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      26.05 ± ∞ ¹   25.98 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4    2.916 ± ∞ ¹   2.796 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                        21.59         21.14        -2.06%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         129.1k ± ∞ ¹      115.1k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_SmallBatch-4                                          534.2k ± ∞ ¹      516.6k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                          632.2k ± ∞ ¹      628.5k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           478.0k ± ∞ ¹      627.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                           640.0k ± ∞ ¹      550.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                                 13.14k ± ∞ ¹      13.15k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          71.88k ± ∞ ¹      68.67k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.814k ± ∞ ¹      9.754k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         65.70k ± ∞ ¹      67.89k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.580k ± ∞ ¹      9.060k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        94.80k ± ∞ ¹     102.00k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.700k ± ∞ ¹      9.600k ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_Concurrent-4                                          115.4k ± ∞ ¹      111.5k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                116.1k ± ∞ ¹      107.8k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   334.0k ± ∞ ¹      333.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                 36.49k ± ∞ ¹      34.99k ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                     87.45k            86.36k        -1.24%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     875.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.729Ki ± ∞ ¹   8.658Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.40Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.4Ki ± ∞ ¹   835.4Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              6.602Ki ± ∞ ¹   6.524Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.251Ki ± ∞ ¹   1.181Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        942.0 ± ∞ ¹     871.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.88Ki ± ∞ ¹   11.92Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.919Ki ± ∞ ¹   8.840Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.4Ki ± ∞ ¹   118.5Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     444.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.461Ki ± ∞ ¹   2.250Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.690Ki ± ∞ ¹   3.566Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   372.0 ± ∞ ¹     372.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
geomean                                                  2.113Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                36.00 ± ∞ ¹   30.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.239                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.543 ± ∞ ¹     3.541 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.078 ± ∞ ¹     5.062 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.984 ± ∞ ¹     4.708 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.052 ± ∞ ¹     4.997 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                    4.613           4.532        -1.77%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.052 ± ∞ ¹   5.056 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.214 ± ∞ ¹   8.164 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    8.018 ± ∞ ¹   7.583 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.142 ± ∞ ¹   8.058 ± ∞ ¹       ~ (p=0.300 n=3) ²
geomean                                                  7.214         7.087        -1.77%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   701.2m ± ∞ ¹        700.4m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        617.7m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         621.4m ± ∞ ¹        620.9m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.2m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=0.800 n=3) ²
geomean                                                       639.5m              638.9m        -0.10%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.14k ± ∞ ¹         13.15k ± ∞ ¹  ~ (p=0.700 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T03:11:50Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │              new.bench               │
                                                       │    sec/op    │    sec/op     vs base                │
Write_SingleEntry-4                                      814.3n ± ∞ ¹   881.2n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       2.030µ ± ∞ ¹   2.086µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       24.15µ ± ∞ ¹   22.73µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                        245.3µ ± ∞ ¹   306.4µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                        3.194m ± ∞ ¹   2.979m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              80.86µ ± ∞ ¹   80.96µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.431µ ± ∞ ¹   1.511µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.19µ ± ∞ ¹   12.02µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      19.71µ ± ∞ ¹   20.59µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    114.3µ ± ∞ ¹   114.5µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     182.2µ ± ∞ ¹   179.1µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.181m ± ∞ ¹   1.194m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       951.0n ± ∞ ¹   997.6n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.025µ ± ∞ ¹   1.081µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentBatched-4                                1.594µ ± ∞ ¹   1.635µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              3.046µ ± ∞ ¹   3.031µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Consumer_SequentialRead-4                                163.2µ ± ∞ ¹   161.8µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_RandomAccess-4                                  149.2µ ± ∞ ¹   155.9µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Consumer_LagMeasurement-4                                258.2n ± ∞ ¹   237.8n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.471µ ± ∞ ¹   3.306µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  16.21µ         16.74µ        +3.28%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.30 ± ∞ ¹   13.78 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4             52.65 ± ∞ ¹   43.96 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4             55.57 ± ∞ ¹   77.31 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              66.80 ± ∞ ¹   58.63 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4              63.96 ± ∞ ¹   54.37 ± ∞ ¹       ~ (p=0.500 n=3) ²
Write_Concurrent-4             9.093 ± ∞ ¹   8.614 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_ConcurrentMultiShard-4   9.147 ± ∞ ¹   8.587 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.15 ± ∞ ¹   24.28 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4    2.881 ± ∞ ¹   2.928 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                        22.04         20.97        -4.85%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         140.5k ± ∞ ¹      118.8k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                          598.3k ± ∞ ¹      499.5k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                          568.2k ± ∞ ¹      790.5k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           638.0k ± ∞ ¹      560.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                           600.0k ± ∞ ¹      510.0k ± ∞ ¹       ~ (p=0.500 n=3) ²
Write_CompressibleEntries-4                                 13.23k ± ∞ ¹      13.08k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          71.91k ± ∞ ¹      69.01k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.465k ± ∞ ¹      9.811k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         128.3k ± ∞ ¹      131.7k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       8.960k ± ∞ ¹      9.430k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        121.2k ± ∞ ¹      108.4k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.800k ± ∞ ¹      9.300k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_Concurrent-4                                          112.3k ± ∞ ¹      106.3k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_ConcurrentMultiShard-4                                110.2k ± ∞ ¹      103.5k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   322.4k ± ∞ ¹      311.3k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                 36.05k ± ∞ ¹      36.64k ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                     93.19k            90.03k        -3.38%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     872.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.727Ki ± ∞ ¹   8.656Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.3Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              6.593Ki ± ∞ ¹   6.537Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.256Ki ± ∞ ¹   1.180Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        940.0 ± ∞ ¹     868.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    12.01Ki ± ∞ ¹   11.84Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.913Ki ± ∞ ¹   8.844Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.3Ki ± ∞ ¹   119.1Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     444.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.646Ki ± ∞ ¹   2.329Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.711Ki ± ∞ ¹   3.567Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   431.0 ± ∞ ¹     456.0 ± ∞ ¹  ~ (p=0.400 n=3) ²
geomean                                                  2.138Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                38.00 ± ∞ ¹   31.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.264                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.583 ± ∞ ¹     3.530 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       4.903 ± ∞ ¹     5.082 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.657 ± ∞ ¹     4.903 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.103 ± ∞ ¹     4.848 ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                    4.520           4.544        +0.53%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.087 ± ∞ ¹   5.029 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     7.922 ± ∞ ¹   8.212 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.500 ± ∞ ¹   7.893 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.226 ± ∞ ¹   7.807 ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                  7.061         7.103        +0.58%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   703.0m ± ∞ ¹        702.0m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.7m ± ∞ ¹        621.1m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.5m ± ∞ ¹        621.0m ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                       639.8m              639.8m        +0.00%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.23k ± ∞ ¹         13.08k ± ∞ ¹  ~ (p=0.400 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T03:18:46Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │               new.bench               │
                                                       │    sec/op    │    sec/op      vs base                │
Write_SingleEntry-4                                      806.6n ± ∞ ¹    894.1n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       1.968µ ± ∞ ¹    2.063µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       24.01µ ± ∞ ¹    21.72µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                        289.6µ ± ∞ ¹    294.3µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                        2.784m ± ∞ ¹    2.728m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              80.16µ ± ∞ ¹    80.82µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.934µ ± ∞ ¹    1.514µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     11.93µ ± ∞ ¹    12.08µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      21.55µ ± ∞ ¹    20.55µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    116.0µ ± ∞ ¹    114.5µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     200.8µ ± ∞ ¹    194.6µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.256m ± ∞ ¹    1.202m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4                                       932.3n ± ∞ ¹   1002.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.004µ ± ∞ ¹    1.069µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.533µ ± ∞ ¹    1.598µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              2.908µ ± ∞ ¹    2.905µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Consumer_SequentialRead-4                                183.6µ ± ∞ ¹    187.8µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Consumer_RandomAccess-4                                  178.0µ ± ∞ ¹    141.1µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                256.1n ± ∞ ¹    241.3n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.020µ ± ∞ ¹    2.661µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  16.60µ          16.49µ        -0.66%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            17.00 ± ∞ ¹   15.07 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_SmallBatch-4             53.51 ± ∞ ¹   50.97 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_LargeBatch-4             65.97 ± ∞ ¹   59.16 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              62.92 ± ∞ ¹   66.38 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              57.56 ± ∞ ¹   54.37 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4             8.988 ± ∞ ¹   8.546 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.439 ± ∞ ¹   8.777 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      28.60 ± ∞ ¹   28.76 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4    3.028 ± ∞ ¹   2.914 ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                        22.69         21.62        -4.75%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         146.6k ± ∞ ¹      129.9k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_SmallBatch-4                                          608.1k ± ∞ ¹      579.2k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_LargeBatch-4                                          674.5k ± ∞ ¹      604.9k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           601.0k ± ∞ ¹      634.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           540.0k ± ∞ ¹      510.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                                 12.86k ± ∞ ¹      12.46k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          70.47k ± ∞ ¹      67.74k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.849k ± ∞ ¹      9.704k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         123.0k ± ∞ ¹      129.6k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.320k ± ∞ ¹     10.210k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        109.8k ± ∞ ¹      109.6k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.800k ± ∞ ¹      9.600k ± ∞ ¹       ~ (p=0.500 n=3) ²
Write_Concurrent-4                                          111.0k ± ∞ ¹      105.5k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                113.7k ± ∞ ¹      105.8k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   366.6k ± ∞ ¹      368.7k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                 37.88k ± ∞ ¹      36.46k ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                     94.08k            91.74k        -2.48%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         951.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.726Ki ± ∞ ¹   8.659Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.4Ki ± ∞ ¹   835.0Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              6.623Ki ± ∞ ¹   6.597Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.250Ki ± ∞ ¹   1.182Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        941.0 ± ∞ ¹     868.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.93Ki ± ∞ ¹   11.71Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.926Ki ± ∞ ¹   8.853Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.2Ki ± ∞ ¹   118.5Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  515.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.715Ki ± ∞ ¹   2.477Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.724Ki ± ∞ ¹   3.559Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   373.0 ± ∞ ¹     374.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
geomean                                                  2.126Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                39.00 ± ∞ ¹   32.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.276                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.453 ± ∞ ¹     3.364 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.102 ± ∞ ¹     5.036 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.845 ± ∞ ¹     5.299 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.106 ± ∞ ¹     4.995 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                    4.569           4.602        +0.71%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              4.945 ± ∞ ¹   4.791 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.244 ± ∞ ¹   8.122 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.801 ± ∞ ¹   8.546 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.226 ± ∞ ¹   8.058 ± ∞ ¹       ~ (p=0.500 n=3) ²
geomean                                                  7.152         7.195        +0.60%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   700.4m ± ∞ ¹        700.9m ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.4m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.3m ± ∞ ¹        620.4m ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                       639.1m              639.5m        +0.06%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         12.86k ± ∞ ¹         12.46k ± ∞ ¹  ~ (p=0.700 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T03:25:14Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │              new.bench               │
                                                       │    sec/op    │    sec/op     vs base                │
Write_SingleEntry-4                                      820.3n ± ∞ ¹   893.8n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       2.010µ ± ∞ ¹   2.083µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_LargeBatch-4                                       22.62µ ± ∞ ¹   25.19µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                        276.4µ ± ∞ ¹   306.9µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                        3.000m ± ∞ ¹   2.478m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              80.85µ ± ∞ ¹   81.98µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.471µ ± ∞ ¹   1.512µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     11.90µ ± ∞ ¹   12.32µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      15.72µ ± ∞ ¹   18.23µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    115.2µ ± ∞ ¹   116.7µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     190.5µ ± ∞ ¹   161.3µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.202m ± ∞ ¹   1.199m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4                                       945.8n ± ∞ ¹   998.7n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.027µ ± ∞ ¹   1.096µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.611µ ± ∞ ¹   1.632µ ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_ConcurrentReadWrite-4                              2.966µ ± ∞ ¹   3.005µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_SequentialRead-4                                165.7µ ± ∞ ¹   184.2µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_RandomAccess-4                                  174.7µ ± ∞ ¹   176.8µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Consumer_LagMeasurement-4                                255.6n ± ∞ ¹   239.4n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.309µ ± ∞ ¹   2.894µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  16.14µ         16.66µ        +3.23%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.75 ± ∞ ¹   15.40 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4             44.91 ± ∞ ¹   51.84 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4             75.96 ± ∞ ¹   58.15 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              58.52 ± ∞ ¹   51.09 ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_MegaBatch-4              57.56 ± ∞ ¹   58.63 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4             8.993 ± ∞ ¹   8.517 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.140 ± ∞ ¹   8.440 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.09 ± ∞ ¹   28.40 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4    2.947 ± ∞ ¹   2.835 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                        21.92         21.03        -4.07%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         144.4k ± ∞ ¹      132.7k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                          510.4k ± ∞ ¹      589.1k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                          776.7k ± ∞ ¹      594.6k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           559.0k ± ∞ ¹      488.0k ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_MegaBatch-4                                           540.0k ± ∞ ¹      550.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                                 13.68k ± ∞ ¹      14.62k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          71.46k ± ∞ ¹      67.83k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.408k ± ∞ ¹      9.586k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         87.14k ± ∞ ¹     120.60k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.800k ± ∞ ¹      8.700k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        115.8k ± ∞ ¹      108.6k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.300k ± ∞ ¹      9.300k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4                                          111.0k ± ∞ ¹      105.1k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                110.1k ± ∞ ¹      101.7k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   321.6k ± ∞ ¹      364.1k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                 36.87k ± ∞ ¹      35.48k ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                     90.76k            89.64k        -1.24%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         948.0 ± ∞ ¹     875.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.726Ki ± ∞ ¹   8.655Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.2Ki ± ∞ ¹   835.3Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              6.545Ki ± ∞ ¹   6.396Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.258Ki ± ∞ ¹   1.185Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        943.0 ± ∞ ¹     870.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.85Ki ± ∞ ¹   11.99Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.910Ki ± ∞ ¹   8.855Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   119.2Ki ± ∞ ¹   119.0Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.692Ki ± ∞ ¹   2.555Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.726Ki ± ∞ ¹   3.582Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   407.0 ± ∞ ¹     409.0 ± ∞ ¹  ~ (p=0.700 n=3) ²
geomean                                                  2.134Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                39.00 ± ∞ ¹   33.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.400 n=3) ²
geomean                                                  9.276                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.684 ± ∞ ¹     3.952 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       4.873 ± ∞ ¹     4.966 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      5.082 ± ∞ ¹     4.520 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     4.842 ± ∞ ¹     4.844 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                    4.585           4.553        -0.69%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.260 ± ∞ ¹   5.621 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     7.874 ± ∞ ¹   8.023 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    8.203 ± ∞ ¹   7.282 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   7.807 ± ∞ ¹   7.807 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                  7.176         7.116        -0.85%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   700.4m ± ∞ ¹        703.0m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         619.6m ± ∞ ¹        621.3m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.4m ± ∞ ¹        620.5m ± ∞ ¹       ~ (p=0.600 n=3) ²
geomean                                                       638.9m              640.0m        +0.17%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.68k ± ∞ ¹         14.62k ± ∞ ¹  ~ (p=0.700 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T03:31:00Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │              new.bench               │
                                                       │    sec/op    │    sec/op     vs base                │
Write_SingleEntry-4                                      852.8n ± ∞ ¹   893.1n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       2.068µ ± ∞ ¹   2.085µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_LargeBatch-4                                       25.90µ ± ∞ ¹   22.70µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                        318.8µ ± ∞ ¹   291.8µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                        2.921m ± ∞ ¹   2.917m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              81.51µ ± ∞ ¹   81.38µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.457µ ± ∞ ¹   1.513µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.39µ ± ∞ ¹   13.02µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      16.22µ ± ∞ ¹   19.70µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    122.9µ ± ∞ ¹   115.4µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     169.8µ ± ∞ ¹   178.7µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.196m ± ∞ ¹   1.186m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       947.8n ± ∞ ¹   998.7n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.022µ ± ∞ ¹   1.080µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.559µ ± ∞ ¹   1.632µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              2.946µ ± ∞ ¹   2.964µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Consumer_SequentialRead-4                                138.8µ ± ∞ ¹   192.8µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  136.5µ ± ∞ ¹   177.3µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                254.8n ± ∞ ¹   249.0n ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_MultiConsumer-4                                 2.057µ ± ∞ ¹   3.233µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  15.94µ         16.98µ        +6.56%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            14.79 ± ∞ ¹   15.36 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4             46.97 ± ∞ ¹   43.94 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4             76.19 ± ∞ ¹   81.29 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4              72.97 ± ∞ ¹   69.20 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              55.43 ± ∞ ¹   61.83 ± ∞ ¹       ~ (p=0.500 n=3) ²
Write_Concurrent-4             9.279 ± ∞ ¹   8.633 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.378 ± ∞ ¹   8.695 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.21 ± ∞ ¹   28.20 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4    2.951 ± ∞ ¹   2.902 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                        22.34         22.44        +0.44%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         127.5k ± ∞ ¹      132.4k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4                                          533.7k ± ∞ ¹      499.3k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                          779.0k ± ∞ ¹      831.2k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                           697.0k ± ∞ ¹      661.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           520.0k ± ∞ ¹      580.0k ± ∞ ¹       ~ (p=0.500 n=3) ²
Write_CompressibleEntries-4                                 14.61k ± ∞ ¹      14.66k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          73.31k ± ∞ ¹      69.20k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.471k ± ∞ ¹      9.669k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         93.61k ± ∞ ¹     118.15k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       8.360k ± ∞ ¹      8.950k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        93.60k ± ∞ ¹     117.70k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.400k ± ∞ ¹      9.800k ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_Concurrent-4                                          114.6k ± ∞ ¹      106.6k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                113.0k ± ∞ ¹      104.8k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   323.2k ± ∞ ¹      361.5k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                 36.92k ± ∞ ¹      36.31k ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                     90.63k            93.97k        +3.68%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         948.0 ± ∞ ¹     872.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.729Ki ± ∞ ¹   8.656Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.5Ki ± ∞ ¹   835.0Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              6.472Ki ± ∞ ¹   6.396Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.257Ki ± ∞ ¹   1.183Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        941.0 ± ∞ ¹     869.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    12.15Ki ± ∞ ¹   11.94Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.908Ki ± ∞ ¹   8.842Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   119.0Ki ± ∞ ¹   118.2Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.433Ki ± ∞ ¹   2.447Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Consumer_RandomAccess-4                                  3.699Ki ± ∞ ¹   3.594Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   377.0 ± ∞ ¹     453.0 ± ∞ ¹  ~ (p=0.200 n=3) ²
geomean                                                  2.115Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                36.00 ± ∞ ¹   32.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.239                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.936 ± ∞ ¹     3.954 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       4.915 ± ∞ ¹     5.018 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.348 ± ∞ ¹     4.649 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     4.897 ± ∞ ¹     5.100 ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                                                    4.505           4.657        +3.38%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.618 ± ∞ ¹   5.637 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     7.927 ± ∞ ¹   8.093 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    6.997 ± ∞ ¹   7.491 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   7.891 ± ∞ ¹   8.226 ± ∞ ¹       ~ (p=0.300 n=3) ²
geomean                                                  7.042         7.282        +3.40%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   701.4m ± ∞ ¹        700.7m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          620.1m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.1m ± ∞ ¹        621.1m ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.6m ± ∞ ¹        620.5m ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                       639.6m              639.7m        +0.01%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         14.61k ± ∞ ¹         14.66k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T03:56:48Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │              new.bench               │
                                                       │    sec/op    │    sec/op     vs base                │
Write_SingleEntry-4                                      808.4n ± ∞ ¹   869.2n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       1.957µ ± ∞ ¹   2.096µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       27.38µ ± ∞ ¹   24.80µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                        270.8µ ± ∞ ¹   319.7µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        3.144m ± ∞ ¹   2.602m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              84.46µ ± ∞ ¹   79.97µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.427µ ± ∞ ¹   1.483µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.53µ ± ∞ ¹   11.88µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      20.48µ ± ∞ ¹   16.71µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    118.1µ ± ∞ ¹   114.7µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     180.0µ ± ∞ ¹   193.1µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.213m ± ∞ ¹   1.184m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4                                       937.3n ± ∞ ¹   978.8n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.002µ ± ∞ ¹   1.066µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.532µ ± ∞ ¹   1.585µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_ConcurrentReadWrite-4                              2.893µ ± ∞ ¹   2.899µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Consumer_SequentialRead-4                                147.1µ ± ∞ ¹   212.7µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  136.0µ ± ∞ ¹   167.8µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                254.3n ± ∞ ¹   243.2n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.190µ ± ∞ ¹   2.600µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  16.10µ         16.60µ        +3.10%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.93 ± ∞ ¹   14.02 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_SmallBatch-4             46.89 ± ∞ ¹   44.44 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_LargeBatch-4             63.30 ± ∞ ¹   56.25 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              70.04 ± ∞ ¹   64.39 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              41.57 ± ∞ ¹   55.43 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4             9.122 ± ∞ ¹   8.936 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.593 ± ∞ ¹   8.656 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.54 ± ∞ ¹   29.78 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_ConcurrentReadWrite-4    3.021 ± ∞ ¹   3.006 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                        21.52         21.20        -1.45%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         145.9k ± ∞ ¹      120.9k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_SmallBatch-4                                          532.8k ± ∞ ¹      505.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_LargeBatch-4                                          647.2k ± ∞ ¹      575.2k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           669.0k ± ∞ ¹      615.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           390.0k ± ∞ ¹      520.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                                 12.58k ± ∞ ¹      14.88k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          72.34k ± ∞ ¹      71.78k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.519k ± ∞ ¹      9.866k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         122.5k ± ∞ ¹      134.5k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.160k ± ∞ ¹      9.150k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        120.2k ± ∞ ¹      117.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.400k ± ∞ ¹      9.800k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                          112.6k ± ∞ ¹      110.3k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                115.6k ± ∞ ¹      104.3k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   327.5k ± ∞ ¹      381.8k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_ConcurrentReadWrite-4                                 37.80k ± ∞ ¹      37.61k ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                     91.28k            92.27k        +1.08%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     875.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.726Ki ± ∞ ¹   8.656Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.39Ki ± ∞ ¹   84.31Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.3Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              6.654Ki ± ∞ ¹   6.381Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.256Ki ± ∞ ¹   1.180Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        942.0 ± ∞ ¹     867.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.97Ki ± ∞ ¹   11.89Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.914Ki ± ∞ ¹   8.842Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   119.0Ki ± ∞ ¹   118.2Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.454Ki ± ∞ ¹   2.728Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Consumer_RandomAccess-4                                  3.702Ki ± ∞ ¹   3.590Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   392.0 ± ∞ ¹     372.0 ± ∞ ¹  ~ (p=0.700 n=3) ²
geomean                                                  2.121Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                36.00 ± ∞ ¹   34.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.239                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.394 ± ∞ ¹     4.012 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       4.940 ± ∞ ¹     5.130 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.751 ± ∞ ¹     4.763 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     4.897 ± ∞ ¹     5.107 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                    4.444           4.730        +6.44%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              4.837 ± ∞ ¹   5.721 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     7.967 ± ∞ ¹   8.258 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.667 ± ∞ ¹   7.659 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   7.891 ± ∞ ¹   8.226 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  6.949         7.386        +6.30%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   702.5m ± ∞ ¹        701.4m ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          620.1m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.2m ± ∞ ¹        621.5m ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.6m ± ∞ ¹        621.1m ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                       639.9m              640.1m        +0.03%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         12.58k ± ∞ ¹         14.88k ± ∞ ¹  ~ (p=0.200 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T04:09:20Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │               new.bench               │
                                                       │    sec/op    │    sec/op      vs base                │
Write_SingleEntry-4                                      826.5n ± ∞ ¹    875.0n ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4                                       2.158µ ± ∞ ¹    2.054µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_LargeBatch-4                                       22.99µ ± ∞ ¹    24.78µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                        288.5µ ± ∞ ¹    272.3µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                        2.712m ± ∞ ¹    2.798m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              81.83µ ± ∞ ¹    89.96µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.443µ ± ∞ ¹    1.523µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     13.41µ ± ∞ ¹    13.55µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      18.43µ ± ∞ ¹    19.22µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    116.4µ ± ∞ ¹    130.5µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     192.1µ ± ∞ ¹    184.8µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.213m ± ∞ ¹    1.365m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       957.4n ± ∞ ¹   1014.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.022µ ± ∞ ¹    1.098µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.597µ ± ∞ ¹    1.681µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              3.100µ ± ∞ ¹    3.149µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_SequentialRead-4                                145.0µ ± ∞ ¹    151.2µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_RandomAccess-4                                  143.4µ ± ∞ ¹    137.0µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Consumer_LagMeasurement-4                                257.5n ± ∞ ¹    239.4n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.363µ ± ∞ ¹    2.993µ ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                  16.20µ          16.87µ        +4.12%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.48 ± ∞ ¹   15.79 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4             44.93 ± ∞ ¹   44.24 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4             78.34 ± ∞ ¹   75.65 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              54.96 ± ∞ ¹   59.47 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              52.23 ± ∞ ¹   54.37 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_Concurrent-4             9.190 ± ∞ ¹   8.658 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.389 ± ∞ ¹   8.784 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      24.70 ± ∞ ¹   24.18 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4    2.910 ± ∞ ¹   2.804 ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                        21.62         21.24        -1.75%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         142.0k ± ∞ ¹      136.1k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4                                          510.5k ± ∞ ¹      502.8k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                          801.0k ± ∞ ¹      773.5k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           525.0k ± ∞ ¹      568.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           490.0k ± ∞ ¹      510.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                                 12.98k ± ∞ ¹      13.23k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          71.15k ± ∞ ¹      68.16k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.610k ± ∞ ¹      8.474k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         99.88k ± ∞ ¹     107.49k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.240k ± ∞ ¹      7.810k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       117.00k ± ∞ ¹      96.30k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.200k ± ∞ ¹      8.400k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                          113.5k ± ∞ ¹      106.9k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                113.1k ± ∞ ¹      105.8k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   316.7k ± ∞ ¹      310.0k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                 36.41k ± ∞ ¹      35.09k ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                     90.29k            86.48k        -4.21%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         948.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.728Ki ± ∞ ¹   8.656Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.39Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.2Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              6.611Ki ± ∞ ¹   6.522Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.254Ki ± ∞ ¹   1.208Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        940.0 ± ∞ ¹     870.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.95Ki ± ∞ ¹   12.22Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.913Ki ± ∞ ¹   8.834Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   119.4Ki ± ∞ ¹   121.1Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     445.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.445Ki ± ∞ ¹   2.299Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.703Ki ± ∞ ¹   3.567Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   417.0 ± ∞ ¹     415.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
geomean                                                  2.127Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                36.00 ± ∞ ¹   31.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.239                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.512 ± ∞ ¹     3.580 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       4.978 ± ∞ ¹     4.390 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.807 ± ∞ ¹     4.055 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     4.787 ± ∞ ¹     4.379 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                    4.479           4.087        -8.74%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              4.991 ± ∞ ¹   5.087 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.044 ± ∞ ¹   7.093 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.734 ± ∞ ¹   6.537 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   7.723 ± ∞ ¹   7.051 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  6.998         6.386        -8.74%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   702.0m ± ∞ ¹        702.2m ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         621.3m ± ∞ ¹        620.5m ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.7m ± ∞ ¹        620.9m ± ∞ ¹       ~ (p=0.800 n=3) ²
geomean                                                       639.8m              639.7m        -0.02%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         12.98k ± ∞ ¹         13.23k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T04:16:44Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │              new.bench               │
                                                       │    sec/op    │    sec/op     vs base                │
Write_SingleEntry-4                                      818.6n ± ∞ ¹   880.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       2.124µ ± ∞ ¹   2.101µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_LargeBatch-4                                       20.64µ ± ∞ ¹   44.63µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_HugeBatch-4                                        277.0µ ± ∞ ¹   358.0µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                        2.782m ± ∞ ¹   2.970m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              82.41µ ± ∞ ¹   81.26µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.455µ ± ∞ ¹   1.521µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.15µ ± ∞ ¹   12.13µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      21.93µ ± ∞ ¹   17.47µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    118.5µ ± ∞ ¹   118.1µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     105.6µ ± ∞ ¹   180.3µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.282m ± ∞ ¹   1.211m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_Concurrent-4                                       945.0n ± ∞ ¹   993.4n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.021µ ± ∞ ¹   1.089µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.579µ ± ∞ ¹   1.601µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              2.961µ ± ∞ ¹   2.994µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Consumer_SequentialRead-4                                136.6µ ± ∞ ¹   148.4µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Consumer_RandomAccess-4                                  133.9µ ± ∞ ¹   142.2µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                256.2n ± ∞ ¹   242.2n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.226µ ± ∞ ¹   2.731µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  15.53µ         17.05µ        +9.76%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench               │
                             │ MB_written  │ MB_written   vs base                 │
Write_SingleEntry-4            16.85 ± ∞ ¹   13.66 ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_SmallBatch-4             45.73 ± ∞ ¹   43.83 ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_LargeBatch-4             64.90 ± ∞ ¹   43.99 ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_HugeBatch-4              70.56 ± ∞ ¹   31.72 ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_MegaBatch-4              58.63 ± ∞ ¹   55.43 ± ∞ ¹        ~ (p=0.500 n=3) ²
Write_Concurrent-4             9.107 ± ∞ ¹   8.692 ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.271 ± ∞ ¹   8.808 ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.45 ± ∞ ¹   24.66 ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_ConcurrentReadWrite-4    2.928 ± ∞ ¹   2.963 ± ∞ ¹        ~ (p=0.400 n=3) ²
geomean                        22.18         18.54        -16.42%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                 │
                                                       │ entries_written │ entries_written  vs base                 │
Write_SingleEntry-4                                         145.3k ± ∞ ¹      117.8k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                          519.6k ± ∞ ¹      498.0k ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                          663.6k ± ∞ ¹      449.8k ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           674.0k ± ∞ ¹      303.0k ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_MegaBatch-4                                           550.0k ± ∞ ¹      520.0k ± ∞ ¹        ~ (p=0.500 n=3) ²
Write_CompressibleEntries-4                                 14.10k ± ∞ ¹      12.49k ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          71.40k ± ∞ ¹      70.23k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.681k ± ∞ ¹      9.704k ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         90.58k ± ∞ ¹      91.03k ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       8.740k ± ∞ ¹      9.210k ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       105.10k ± ∞ ¹      78.60k ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.700k ± ∞ ¹      9.700k ± ∞ ¹        ~ (p=0.600 n=3) ²
Write_Concurrent-4                                          112.4k ± ∞ ¹      107.3k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                111.7k ± ∞ ¹      106.1k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   326.3k ± ∞ ¹      316.1k ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_ConcurrentReadWrite-4                                 36.63k ± ∞ ¹      37.08k ± ∞ ¹        ~ (p=0.400 n=3) ²
geomean                                                     90.95k            80.35k        -11.66%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     872.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.726Ki ± ∞ ¹   8.657Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.39Ki ± ∞ ¹   84.43Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                        835.2Ki ± ∞ ¹   835.1Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              6.508Ki ± ∞ ¹   6.594Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.253Ki ± ∞ ¹   1.182Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        942.0 ± ∞ ¹     870.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    12.06Ki ± ∞ ¹   11.89Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.934Ki ± ∞ ¹   8.834Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.5Ki ± ∞ ¹   118.4Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     444.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.513Ki ± ∞ ¹   2.346Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.703Ki ± ∞ ¹   3.561Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   398.0 ± ∞ ¹     384.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
geomean                                                  2.123Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                37.00 ± ∞ ¹   31.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.252                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.799 ± ∞ ¹     3.374 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.015 ± ∞ ¹     5.036 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.535 ± ∞ ¹     4.785 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.049 ± ∞ ¹     5.056 ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                    4.570           4.503        -1.47%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.421 ± ∞ ¹   4.802 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.103 ± ∞ ¹   8.122 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.315 ± ∞ ¹   7.709 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.142 ± ∞ ¹   8.142 ± ∞ ¹       ~ (p=0.600 n=3) ²
geomean                                                  7.152         7.034        -1.65%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   701.2m ± ∞ ¹        702.5m ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.2m ± ∞ ¹        620.9m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.1m ± ∞ ¹        620.9m ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                       639.2m              640.2m        +0.16%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         14.10k ± ∞ ¹         12.49k ± ∞ ¹  ~ (p=0.700 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T04:23:25Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │              new.bench               │
                                                       │    sec/op    │    sec/op     vs base                │
Write_SingleEntry-4                                      808.9n ± ∞ ¹   872.5n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       2.041µ ± ∞ ¹   2.033µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                       23.23µ ± ∞ ¹   18.47µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                        332.7µ ± ∞ ¹   289.5µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                        2.783m ± ∞ ¹   2.730m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              90.11µ ± ∞ ¹   82.07µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.461µ ± ∞ ¹   1.519µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.19µ ± ∞ ¹   12.05µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      17.36µ ± ∞ ¹   17.75µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    117.5µ ± ∞ ¹   116.2µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     186.1µ ± ∞ ¹   163.0µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.214m ± ∞ ¹   1.194m ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_Concurrent-4                                       959.6n ± ∞ ¹   985.1n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.048µ ± ∞ ¹   1.076µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.619µ ± ∞ ¹   1.569µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              3.034µ ± ∞ ¹   3.053µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                184.0µ ± ∞ ¹   188.1µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  186.8µ ± ∞ ¹   179.9µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                258.4n ± ∞ ¹   233.9n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.279µ ± ∞ ¹   3.268µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  16.63µ         16.43µ        -1.23%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.98 ± ∞ ¹   15.59 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4             44.87 ± ∞ ¹   43.75 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4             76.49 ± ∞ ¹   77.43 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              63.24 ± ∞ ¹   66.06 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              54.37 ± ∞ ¹   61.83 ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_Concurrent-4             9.180 ± ∞ ¹   8.896 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_ConcurrentMultiShard-4   9.169 ± ∞ ¹   8.683 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentBatched-4      25.53 ± ∞ ¹   29.61 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4    2.826 ± ∞ ¹   2.942 ± ∞ ¹       ~ (p=0.300 n=3) ²
geomean                        22.02         22.45        +1.98%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         146.4k ± ∞ ¹      134.4k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                          509.9k ± ∞ ¹      497.2k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                          782.1k ± ∞ ¹      791.7k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           604.0k ± ∞ ¹      631.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           510.0k ± ∞ ¹      580.0k ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_CompressibleEntries-4                                 14.59k ± ∞ ¹      14.47k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          72.96k ± ∞ ¹      67.39k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.804k ± ∞ ¹      9.564k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         78.91k ± ∞ ¹     128.54k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.200k ± ∞ ¹      9.320k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       116.60k ± ∞ ¹      84.00k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.700k ± ∞ ¹      9.700k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4                                          113.3k ± ∞ ¹      109.8k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_ConcurrentMultiShard-4                                110.5k ± ∞ ¹      104.6k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentBatched-4                                   327.3k ± ∞ ¹      379.7k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                 35.37k ± ∞ ¹      36.81k ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                                                     91.06k            92.42k        +1.49%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     89.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         948.0 ± ∞ ¹     872.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.723Ki ± ∞ ¹   8.656Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.39Ki ± ∞ ¹   84.33Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.0Ki ± ∞ ¹   835.0Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              6.474Ki ± ∞ ¹   6.411Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.251Ki ± ∞ ¹   1.185Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        940.0 ± ∞ ¹     868.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.95Ki ± ∞ ¹   11.87Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.918Ki ± ∞ ¹   8.848Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.5Ki ± ∞ ¹   118.4Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.686Ki ± ∞ ¹   2.487Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.732Ki ± ∞ ¹   3.603Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   401.0 ± ∞ ¹     458.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  2.130Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                39.00 ± ∞ ¹   32.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.276                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.928 ± ∞ ¹     3.905 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.078 ± ∞ ¹     4.964 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.778 ± ∞ ¹     4.846 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.058 ± ∞ ¹     5.052 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                    4.686           4.667        -0.39%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.610 ± ∞ ¹   5.564 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.206 ± ∞ ¹   8.005 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.700 ± ∞ ¹   7.801 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.142 ± ∞ ¹   8.142 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                  7.330         7.293        -0.50%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   700.7m ± ∞ ¹        702.0m ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.5m ± ∞ ¹        620.9m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.4m ± ∞ ¹        620.6m ± ∞ ¹       ~ (p=0.600 n=3) ²
geomean                                                       639.2m              639.7m        +0.07%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         14.59k ± ∞ ¹         14.47k ± ∞ ¹  ~ (p=0.700 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T04:32:15Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │   old.bench   │               new.bench               │
                                                       │    sec/op     │    sec/op     vs base                 │
Write_SingleEntry-4                                       836.7n ± ∞ ¹   830.2n ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_SmallBatch-4                                        2.000µ ± ∞ ¹   1.769µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                        30.22µ ± ∞ ¹   11.36µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                         309.6µ ± ∞ ¹   114.8µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                         2.668m ± ∞ ¹   1.272m ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                               81.62µ ± ∞ ¹   79.12µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        1.477µ ± ∞ ¹   1.290µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4      11.94µ ± ∞ ¹   11.60µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      18.717µ ± ∞ ¹   6.637µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4     114.6µ ± ∞ ¹   109.8µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     178.98µ ± ∞ ¹   61.68µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4    1.206m ± ∞ ¹   1.158m ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        942.0n ± ∞ ¹   957.3n ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_ConcurrentMultiShard-4                             1014.0n ± ∞ ¹   997.9n ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_ConcurrentBatched-4                                 1.563µ ± ∞ ¹   1.460µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                               2.911µ ± ∞ ¹   2.875µ ± ∞ ¹        ~ (p=0.200 n=3) ²
Consumer_SequentialRead-4                                 143.2µ ± ∞ ¹   151.9µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Consumer_RandomAccess-4                                   138.3µ ± ∞ ¹   141.1µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Consumer_LagMeasurement-4                                 259.8n ± ∞ ¹   238.7n ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                  2.051µ ± ∞ ¹   2.723µ ± ∞ ¹        ~ (p=0.100 n=3) ²
geomean                                                   16.04µ         12.48µ        -22.22%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │               new.bench               │
                             │ MB_written  │  MB_written   vs base                 │
Write_SingleEntry-4            16.43 ± ∞ ¹    15.52 ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_SmallBatch-4             46.76 ± ∞ ¹    55.04 ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_LargeBatch-4             68.20 ± ∞ ¹   101.70 ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_HugeBatch-4              69.31 ± ∞ ¹    94.54 ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_MegaBatch-4              55.43 ± ∞ ¹    99.14 ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_Concurrent-4             9.247 ± ∞ ¹   10.130 ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.485 ± ∞ ¹    8.954 ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.56 ± ∞ ¹    29.27 ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4    2.973 ± ∞ ¹    3.182 ± ∞ ¹        ~ (p=0.400 n=3) ²
geomean                        22.26          26.69        +19.92%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                 │
                                                       │ entries_written │ entries_written  vs base                 │
Write_SingleEntry-4                                         141.6k ± ∞ ¹      133.8k ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_SmallBatch-4                                          531.3k ± ∞ ¹      625.5k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                          697.3k ± ∞ ¹     1039.6k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                           662.0k ± ∞ ¹      903.0k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                           520.0k ± ∞ ¹      930.0k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                                 12.85k ± ∞ ¹      14.28k ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          73.34k ± ∞ ¹      86.17k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.850k ± ∞ ¹     10.046k ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         77.94k ± ∞ ¹     178.72k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.540k ± ∞ ¹      9.730k ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        119.8k ± ∞ ¹      178.2k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.700k ± ∞ ¹     10.200k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_Concurrent-4                                          114.2k ± ∞ ¹      125.1k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                114.3k ± ∞ ¹      107.9k ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   327.7k ± ∞ ¹      375.2k ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                 37.20k ± ∞ ¹      39.81k ± ∞ ¹        ~ (p=0.400 n=3) ²
geomean                                                     91.23k            111.6k        +22.28%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     878.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.726Ki ± ∞ ¹   8.658Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.33Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.4Ki ± ∞ ¹   835.2Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_CompressibleEntries-4                              6.631Ki ± ∞ ¹   6.429Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.250Ki ± ∞ ¹   1.176Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        941.0 ± ∞ ¹     867.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.89Ki ± ∞ ¹   11.79Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.912Ki ± ∞ ¹   8.815Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.5Ki ± ∞ ¹   117.5Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.554Ki ± ∞ ¹   2.275Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.703Ki ± ∞ ¹   3.566Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   371.0 ± ∞ ¹     384.0 ± ∞ ¹  ~ (p=0.600 n=3) ²
geomean                                                  2.118Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                37.00 ± ∞ ¹   30.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.252                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.466 ± ∞ ¹     3.857 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.112 ± ∞ ¹     5.204 ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.954 ± ∞ ¹     5.057 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.050 ± ∞ ¹     5.317 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                    4.588           4.820        +5.04%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              4.941 ± ∞ ¹   5.491 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.244 ± ∞ ¹   8.409 ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.985 ± ∞ ¹   8.144 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.142 ± ∞ ¹   8.562 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  7.174         7.533        +5.01%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   701.4m ± ∞ ¹        702.2m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          620.1m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.1m ± ∞ ¹        620.2m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.6m ± ∞ ¹        621.0m ± ∞ ¹       ~ (p=0.300 n=3) ²
geomean                                                       639.6m              639.6m        +0.00%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         12.85k ± ∞ ¹         14.28k ± ∞ ¹  ~ (p=0.700 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T04:39:19Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │               new.bench               │
                                                       │    sec/op    │    sec/op      vs base                │
Write_SingleEntry-4                                      801.4n ± ∞ ¹    946.1n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       1.973µ ± ∞ ¹    2.024µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       27.30µ ± ∞ ¹    25.73µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                        274.1µ ± ∞ ¹    282.7µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                        2.943m ± ∞ ¹    2.949m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              81.98µ ± ∞ ¹    80.00µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.447µ ± ∞ ¹    1.507µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.03µ ± ∞ ¹    12.04µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      20.39µ ± ∞ ¹    18.67µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    115.4µ ± ∞ ¹    113.5µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     176.8µ ± ∞ ¹    183.2µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.193m ± ∞ ¹    1.183m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       938.0n ± ∞ ¹   1021.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             989.7n ± ∞ ¹   1086.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.550µ ± ∞ ¹    1.587µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              2.931µ ± ∞ ¹    2.864µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                133.7µ ± ∞ ¹    146.6µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  132.4µ ± ∞ ¹    133.8µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Consumer_LagMeasurement-4                                255.4n ± ∞ ¹    241.4n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.229µ ± ∞ ¹    2.802µ ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                  15.89µ          16.34µ        +2.83%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.17 ± ∞ ¹   13.29 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_SmallBatch-4             47.33 ± ∞ ¹   44.43 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4             61.44 ± ∞ ¹   64.68 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              42.40 ± ∞ ¹   45.65 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4              65.03 ± ∞ ¹   55.43 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4             9.433 ± ∞ ¹   8.564 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.594 ± ∞ ¹   8.736 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      26.12 ± ∞ ¹   29.08 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4    2.880 ± ∞ ¹   3.001 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                        21.25         20.47        -3.66%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         139.4k ± ∞ ¹      114.6k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_SmallBatch-4                                          537.9k ± ∞ ¹      504.9k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                          628.2k ± ∞ ¹      661.4k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           405.0k ± ∞ ¹      436.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                           610.0k ± ∞ ¹      520.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                                 13.33k ± ∞ ¹      14.67k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          72.44k ± ∞ ¹      68.86k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.865k ± ∞ ¹      9.734k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         134.6k ± ∞ ¹      135.9k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.570k ± ∞ ¹      9.150k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        120.3k ± ∞ ¹      120.7k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.800k ± ∞ ¹      9.900k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4                                          116.5k ± ∞ ¹      105.7k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                115.6k ± ∞ ¹      105.3k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   334.9k ± ∞ ¹      372.8k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                 36.04k ± ∞ ¹      37.56k ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                     92.22k            90.36k        -2.02%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         946.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.726Ki ± ∞ ¹   8.658Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.44Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.2Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              6.577Ki ± ∞ ¹   6.397Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.250Ki ± ∞ ¹   1.182Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        941.0 ± ∞ ¹     871.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.88Ki ± ∞ ¹   11.90Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.910Ki ± ∞ ¹   8.846Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.3Ki ± ∞ ¹   118.0Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.392Ki ± ∞ ¹   2.284Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.696Ki ± ∞ ¹   3.564Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   404.0 ± ∞ ¹     394.0 ± ∞ ¹  ~ (p=0.400 n=3) ²
geomean                                                  2.118Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                36.00 ± ∞ ¹   30.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.239                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.588 ± ∞ ¹     3.952 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.110 ± ∞ ¹     5.042 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.964 ± ∞ ¹     4.755 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.107 ± ∞ ¹     5.152 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                    4.643           4.700        +1.23%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.125 ± ∞ ¹   5.641 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.257 ± ∞ ¹   8.147 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    8.010 ± ∞ ¹   7.659 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.226 ± ∞ ¹   8.310 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                  7.267         7.354        +1.20%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   700.1m ± ∞ ¹        702.5m ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.4m ± ∞ ¹        620.9m ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.9m ± ∞ ¹        620.3m ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                       639.2m              639.7m        +0.08%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.33k ± ∞ ¹         14.67k ± ∞ ¹  ~ (p=0.400 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T04:52:56Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │               new.bench               │
                                                       │    sec/op    │    sec/op      vs base                │
Write_SingleEntry-4                                      800.9n ± ∞ ¹    911.1n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       1.956µ ± ∞ ¹    2.067µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       24.38µ ± ∞ ¹    28.82µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                        309.0µ ± ∞ ¹    264.3µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                        2.859m ± ∞ ¹    3.123m ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressibleEntries-4                              81.04µ ± ∞ ¹    80.18µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.425µ ± ∞ ¹    1.563µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.09µ ± ∞ ¹    12.27µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      19.58µ ± ∞ ¹    16.24µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    116.3µ ± ∞ ¹    115.8µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     197.6µ ± ∞ ¹    175.2µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.186m ± ∞ ¹    1.186m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4                                       935.6n ± ∞ ¹   1016.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             997.2n ± ∞ ¹   1081.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.531µ ± ∞ ¹    1.601µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              2.946µ ± ∞ ¹    2.922µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Consumer_SequentialRead-4                                156.5µ ± ∞ ¹    168.8µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Consumer_RandomAccess-4                                  155.0µ ± ∞ ¹    163.6µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Consumer_LagMeasurement-4                                254.2n ± ∞ ¹    236.4n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.248µ ± ∞ ¹    2.556µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  16.16µ          16.53µ        +2.29%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            14.90 ± ∞ ¹   14.72 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4             46.90 ± ∞ ¹   51.15 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4             76.64 ± ∞ ¹   78.47 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              46.07 ± ∞ ¹   68.05 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4              59.70 ± ∞ ¹   55.43 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4             9.111 ± ∞ ¹   8.602 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.737 ± ∞ ¹   8.634 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      26.48 ± ∞ ¹   24.63 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentReadWrite-4    2.970 ± ∞ ¹   2.930 ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                        21.61         21.97        +1.68%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         128.4k ± ∞ ¹      126.9k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4                                          532.9k ± ∞ ¹      581.3k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                          783.6k ± ∞ ¹      802.4k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           440.0k ± ∞ ¹      650.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                           560.0k ± ∞ ¹      520.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                                 13.20k ± ∞ ¹      14.66k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          82.71k ± ∞ ¹      68.32k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.807k ± ∞ ¹      9.626k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        122.30k ± ∞ ¹      86.82k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.490k ± ∞ ¹      9.570k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        111.7k ± ∞ ¹      119.8k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.800k ± ∞ ¹      9.600k ± ∞ ¹       ~ (p=0.500 n=3) ²
Write_Concurrent-4                                          112.5k ± ∞ ¹      106.2k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                117.3k ± ∞ ¹      104.0k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   339.5k ± ∞ ¹      315.8k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentReadWrite-4                                 37.17k ± ∞ ¹      36.66k ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                                                     92.74k            91.36k        -1.48%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.725Ki ± ∞ ¹   8.656Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.42Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.4Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              6.596Ki ± ∞ ¹   6.393Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.251Ki ± ∞ ¹   1.184Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        943.0 ± ∞ ¹     868.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.91Ki ± ∞ ¹   11.81Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.915Ki ± ∞ ¹   8.842Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.2Ki ± ∞ ¹   118.6Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     444.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.589Ki ± ∞ ¹   2.447Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.714Ki ± ∞ ¹   3.586Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   406.0 ± ∞ ¹     364.0 ± ∞ ¹  ~ (p=0.400 n=3) ²
geomean                                                  2.129Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                38.00 ± ∞ ¹   32.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.264                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.561 ± ∞ ¹     3.954 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.080 ± ∞ ¹     4.981 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.938 ± ∞ ¹     4.975 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.110 ± ∞ ¹     5.009 ± ∞ ¹       ~ (p=0.500 n=3) ²
geomean                                                    4.622           4.707        +1.83%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.075 ± ∞ ¹   5.637 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.208 ± ∞ ¹   8.057 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.943 ± ∞ ¹   8.010 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.226 ± ∞ ¹   8.058 ± ∞ ¹       ~ (p=0.500 n=3) ²
geomean                                                  7.223         7.358        +1.87%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   702.5m ± ∞ ¹        703.3m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         621.1m ± ∞ ¹        621.1m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        621.2m ± ∞ ¹        621.4m ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                                                       640.0m              640.2m        +0.04%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.20k ± ∞ ¹         14.66k ± ∞ ¹  ~ (p=0.400 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T04:57:19Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │               new.bench               │
                                                       │    sec/op    │    sec/op      vs base                │
Write_SingleEntry-4                                      805.4n ± ∞ ¹    891.7n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       1.945µ ± ∞ ¹    2.037µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                       22.97µ ± ∞ ¹    23.09µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                        239.4µ ± ∞ ¹    302.8µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                        3.388m ± ∞ ¹    2.962m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              80.05µ ± ∞ ¹    79.81µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.458µ ± ∞ ¹    1.523µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.00µ ± ∞ ¹    11.84µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      18.91µ ± ∞ ¹    20.18µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    114.5µ ± ∞ ¹    115.5µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     180.0µ ± ∞ ¹    161.3µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.188m ± ∞ ¹    1.172m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       935.7n ± ∞ ¹    999.8n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             987.7n ± ∞ ¹   1066.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.527µ ± ∞ ¹    1.577µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              2.875µ ± ∞ ¹    2.861µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_SequentialRead-4                                133.8µ ± ∞ ¹    139.9µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Consumer_RandomAccess-4                                  129.2µ ± ∞ ¹    132.8µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Consumer_LagMeasurement-4                                254.8n ± ∞ ¹    236.6n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 1.941µ ± ∞ ¹    2.477µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  15.52µ          16.03µ        +3.27%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            14.72 ± ∞ ¹   15.27 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_SmallBatch-4             47.52 ± ∞ ¹   44.08 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_LargeBatch-4             62.03 ± ∞ ¹   81.94 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4              70.46 ± ∞ ¹   65.43 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              63.96 ± ∞ ¹   63.96 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4             9.388 ± ∞ ¹   8.899 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.854 ± ∞ ¹   9.168 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      26.30 ± ∞ ¹   29.99 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4    3.077 ± ∞ ¹   3.003 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                        22.48         22.85        +1.65%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         126.9k ± ∞ ¹      131.6k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_SmallBatch-4                                          540.0k ± ∞ ¹      500.9k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_LargeBatch-4                                          634.3k ± ∞ ¹      837.8k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                           673.0k ± ∞ ¹      625.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           600.0k ± ∞ ¹      600.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                                 13.53k ± ∞ ¹      13.08k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          72.42k ± ∞ ¹      68.50k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.610k ± ∞ ¹      9.589k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         77.30k ± ∞ ¹     131.84k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       8.890k ± ∞ ¹      9.340k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       113.80k ± ∞ ¹      86.70k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.700k ± ∞ ¹      9.900k ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_Concurrent-4                                          115.9k ± ∞ ¹      109.9k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                118.7k ± ∞ ¹      110.5k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   337.1k ± ∞ ¹      384.5k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                 38.51k ± ∞ ¹      37.58k ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                     91.07k            93.30k        +2.45%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     89.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         946.0 ± ∞ ¹     875.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.727Ki ± ∞ ¹   8.655Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.39Ki ± ∞ ¹   84.34Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.4Ki ± ∞ ¹   835.3Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              6.564Ki ± ∞ ¹   6.535Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.255Ki ± ∞ ¹   1.185Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        943.0 ± ∞ ¹     868.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    12.03Ki ± ∞ ¹   11.87Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.919Ki ± ∞ ¹   8.852Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.5Ki ± ∞ ¹   118.0Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.406Ki ± ∞ ¹   2.238Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.699Ki ± ∞ ¹   3.562Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   361.0 ± ∞ ¹     353.0 ± ∞ ¹  ~ (p=0.700 n=3) ²
geomean                                                  2.109Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                36.00 ± ∞ ¹   30.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.239                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.653 ± ∞ ¹     3.530 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       4.988 ± ∞ ¹     4.958 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.617 ± ∞ ¹     4.847 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.052 ± ∞ ¹     5.160 ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                    4.540           4.574        +0.74%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.202 ± ∞ ¹   5.029 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.044 ± ∞ ¹   8.026 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.441 ± ∞ ¹   7.818 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.142 ± ∞ ¹   8.310 ± ∞ ¹       ~ (p=0.300 n=3) ²
geomean                                                  7.096         7.156        +0.85%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   702.2m ± ∞ ¹        702.0m ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          620.1m ± ∞ ¹        617.7m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.1m ± ∞ ¹        620.7m ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.5m ± ∞ ¹        620.6m ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                       639.8m              639.3m        -0.08%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.53k ± ∞ ¹         13.08k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T05:32:27Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │               new.bench               │
                                                       │    sec/op    │    sec/op      vs base                │
Write_SingleEntry-4                                      800.2n ± ∞ ¹    886.1n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       1.983µ ± ∞ ¹    2.012µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       27.73µ ± ∞ ¹    25.13µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_HugeBatch-4                                        272.6µ ± ∞ ¹    248.3µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                        2.774m ± ∞ ¹    3.503m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              80.77µ ± ∞ ¹    81.38µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.429µ ± ∞ ¹    1.501µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     11.86µ ± ∞ ¹    12.13µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      18.86µ ± ∞ ¹    16.53µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    116.6µ ± ∞ ¹    113.6µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     193.9µ ± ∞ ¹    177.2µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.192m ± ∞ ¹    1.177m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       934.1n ± ∞ ¹    989.1n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             993.4n ± ∞ ¹   1070.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.536µ ± ∞ ¹    1.563µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              2.931µ ± ∞ ¹    2.937µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_SequentialRead-4                                155.2µ ± ∞ ¹    181.1µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  143.9µ ± ∞ ¹    146.3µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Consumer_LagMeasurement-4                                255.2n ± ∞ ¹    241.2n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.071µ ± ∞ ¹    2.620µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  15.95µ          16.34µ        +2.42%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            14.62 ± ∞ ¹   13.69 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4             47.47 ± ∞ ¹   44.74 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_LargeBatch-4             65.35 ± ∞ ¹   60.04 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4              49.00 ± ∞ ¹   70.56 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              52.23 ± ∞ ¹   59.70 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4             9.276 ± ∞ ¹   8.862 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentMultiShard-4   9.661 ± ∞ ¹   9.014 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      26.50 ± ∞ ¹   30.26 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4    2.990 ± ∞ ¹   2.967 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                        21.09         21.80        +3.37%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         126.1k ± ∞ ¹      118.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4                                          539.4k ± ∞ ¹      508.4k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_LargeBatch-4                                          668.2k ± ∞ ¹      613.9k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                           468.0k ± ∞ ¹      674.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           490.0k ± ∞ ¹      560.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                                 13.24k ± ∞ ¹      12.97k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          72.97k ± ∞ ¹      70.66k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.840k ± ∞ ¹      8.359k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        105.74k ± ∞ ¹      85.14k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.960k ± ∞ ¹      9.030k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        111.8k ± ∞ ¹      116.8k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.000k ± ∞ ¹      9.800k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_Concurrent-4                                          114.5k ± ∞ ¹      109.4k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentMultiShard-4                                116.4k ± ∞ ¹      108.6k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   339.7k ± ∞ ¹      387.9k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                 37.41k ± ∞ ¹      37.13k ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                     89.77k            89.19k        -0.64%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.729Ki ± ∞ ¹   8.656Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.40Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_MegaBatch-4                                        835.4Ki ± ∞ ¹   835.2Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              6.592Ki ± ∞ ¹   6.549Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.249Ki ± ∞ ¹   1.210Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        943.0 ± ∞ ¹     869.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.82Ki ± ∞ ¹   11.92Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.922Ki ± ∞ ¹   8.837Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   119.8Ki ± ∞ ¹   118.2Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     442.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.632Ki ± ∞ ¹   2.492Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Consumer_RandomAccess-4                                  3.701Ki ± ∞ ¹   3.578Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   375.0 ± ∞ ¹     373.0 ± ∞ ¹  ~ (p=0.700 n=3) ²
geomean                                                  2.122Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                38.00 ± ∞ ¹   32.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.264                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.573 ± ∞ ¹     3.508 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.087 ± ∞ ¹     4.330 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      5.171 ± ∞ ¹     4.684 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     4.691 ± ∞ ¹     5.103 ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                    4.582           4.365        -4.74%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.091 ± ∞ ¹   4.987 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.236 ± ∞ ¹   6.996 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    8.337 ± ∞ ¹   7.558 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   7.555 ± ∞ ¹   8.226 ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                  7.169         6.825        -4.80%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   702.0m ± ∞ ¹        701.4m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.3m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.9m ± ∞ ¹        620.4m ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                       639.6m              639.3m        -0.05%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.24k ± ∞ ¹         12.97k ± ∞ ¹  ~ (p=0.700 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T05:37:20Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │               new.bench               │
                                                       │    sec/op    │    sec/op      vs base                │
Write_SingleEntry-4                                      809.2n ± ∞ ¹    909.8n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       1.969µ ± ∞ ¹    2.053µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       24.20µ ± ∞ ¹    28.84µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_HugeBatch-4                                        312.7µ ± ∞ ¹    259.0µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                        2.994m ± ∞ ¹    3.219m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              81.23µ ± ∞ ¹    80.88µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.445µ ± ∞ ¹    1.543µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     11.98µ ± ∞ ¹    12.11µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      20.54µ ± ∞ ¹    19.36µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    115.4µ ± ∞ ¹    113.7µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     182.6µ ± ∞ ¹    199.8µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.194m ± ∞ ¹    1.198m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_Concurrent-4                                       941.5n ± ∞ ¹   1024.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.006µ ± ∞ ¹    1.103µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.569µ ± ∞ ¹    1.605µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              2.876µ ± ∞ ¹    2.917µ ± ∞ ¹       ~ (p=0.300 n=3) ²
Consumer_SequentialRead-4                                132.1µ ± ∞ ¹    128.3µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Consumer_RandomAccess-4                                  125.1µ ± ∞ ¹    117.8µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Consumer_LagMeasurement-4                                254.5n ± ∞ ¹    236.4n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.240µ ± ∞ ¹    2.657µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  15.89µ          16.32µ        +2.69%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.27 ± ∞ ¹   15.13 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_SmallBatch-4             46.44 ± ∞ ¹   43.17 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4             61.51 ± ∞ ¹   49.00 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              61.35 ± ∞ ¹   70.77 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4              55.43 ± ∞ ¹   58.63 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4             9.335 ± ∞ ¹   8.459 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.636 ± ∞ ¹   8.821 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.63 ± ∞ ¹   24.89 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4    2.988 ± ∞ ¹   3.068 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                        21.75         20.89        -3.96%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         140.3k ± ∞ ¹      130.4k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_SmallBatch-4                                          527.7k ± ∞ ¹      490.5k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                          628.9k ± ∞ ¹      501.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           586.0k ± ∞ ¹      676.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                           520.0k ± ∞ ¹      550.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                                 13.41k ± ∞ ¹      14.77k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          73.27k ± ∞ ¹      69.88k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.885k ± ∞ ¹      9.824k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         91.59k ± ∞ ¹     126.85k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.190k ± ∞ ¹      9.280k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        115.4k ± ∞ ¹      117.8k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.800k ± ∞ ¹      9.800k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4                                          115.3k ± ∞ ¹      104.4k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                116.1k ± ∞ ¹      106.3k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   328.5k ± ∞ ¹      319.1k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                 37.38k ± ∞ ¹      38.40k ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                     90.85k            91.05k        +0.22%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     874.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.728Ki ± ∞ ¹   8.657Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.5Ki ± ∞ ¹   835.3Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              6.570Ki ± ∞ ¹   6.389Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.249Ki ± ∞ ¹   1.181Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        942.0 ± ∞ ¹     869.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.96Ki ± ∞ ¹   11.88Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.912Ki ± ∞ ¹   8.842Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.3Ki ± ∞ ¹   118.2Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     444.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.392Ki ± ∞ ¹   2.223Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.693Ki ± ∞ ¹   3.547Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   403.0 ± ∞ ¹     375.0 ± ∞ ¹  ~ (p=0.500 n=3) ²
geomean                                                  2.119Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                36.00 ± ∞ ¹   30.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.239                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.622 ± ∞ ¹     3.986 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.120 ± ∞ ¹     5.079 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.777 ± ∞ ¹     4.814 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.106 ± ∞ ¹     5.100 ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                                                    4.612           4.722        +2.38%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.156 ± ∞ ¹   5.679 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.274 ± ∞ ¹   8.223 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.692 ± ∞ ¹   7.767 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.226 ± ∞ ¹   8.226 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                  7.208         7.391        +2.53%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   702.5m ± ∞ ¹        701.2m ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.8m ± ∞ ¹        620.0m ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.3m ± ∞ ¹        620.0m ± ∞ ¹       ~ (p=0.300 n=3) ²
geomean                                                       639.7m              639.1m        -0.09%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.41k ± ∞ ¹         14.77k ± ∞ ¹  ~ (p=0.100 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T05:46:25Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │               new.bench               │
                                                       │    sec/op    │    sec/op      vs base                │
Write_SingleEntry-4                                      808.8n ± ∞ ¹    910.4n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       1.990µ ± ∞ ¹    2.084µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       24.53µ ± ∞ ¹    24.46µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                        269.1µ ± ∞ ¹    281.7µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                        2.737m ± ∞ ¹    3.073m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              81.29µ ± ∞ ¹    81.15µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.431µ ± ∞ ¹    1.572µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     11.98µ ± ∞ ¹    12.06µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      18.54µ ± ∞ ¹    15.83µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    114.7µ ± ∞ ¹    114.0µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     182.6µ ± ∞ ¹    146.4µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.206m ± ∞ ¹    1.183m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       944.7n ± ∞ ¹   1019.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.010µ ± ∞ ¹    1.091µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.536µ ± ∞ ¹    1.593µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              2.908µ ± ∞ ¹    2.912µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_SequentialRead-4                                142.4µ ± ∞ ¹    139.2µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_RandomAccess-4                                  136.9µ ± ∞ ¹    127.3µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                254.9n ± ∞ ¹    240.4n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.013µ ± ∞ ¹    2.550µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  15.68µ          15.91µ        +1.46%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            14.90 ± ∞ ¹   13.53 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4             47.06 ± ∞ ¹   43.33 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4             79.52 ± ∞ ¹   81.96 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_HugeBatch-4              59.36 ± ∞ ¹   62.92 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              58.63 ± ∞ ¹   60.76 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4             9.026 ± ∞ ¹   8.463 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.537 ± ∞ ¹   8.817 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.86 ± ∞ ¹   25.30 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentReadWrite-4    3.056 ± ∞ ¹   2.972 ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                        22.22         21.62        -2.71%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         128.4k ± ∞ ¹      116.6k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                          534.8k ± ∞ ¹      492.3k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                          813.1k ± ∞ ¹      838.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_HugeBatch-4                                           567.0k ± ∞ ¹      601.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           550.0k ± ∞ ¹      570.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                                 13.22k ± ∞ ¹      14.41k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          71.67k ± ∞ ¹      65.55k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.853k ± ∞ ¹      9.193k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         99.97k ± ∞ ¹      89.68k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.260k ± ∞ ¹      9.320k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        119.5k ± ∞ ¹      117.1k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.700k ± ∞ ¹      9.800k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4                                          111.4k ± ∞ ¹      104.5k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                114.9k ± ∞ ¹      106.2k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   331.6k ± ∞ ¹      324.4k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentReadWrite-4                                 38.24k ± ∞ ¹      37.19k ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                     92.42k            89.95k        -2.67%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     872.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.726Ki ± ∞ ¹   8.656Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.34Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.0Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              6.588Ki ± ∞ ¹   6.418Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.250Ki ± ∞ ¹   1.192Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        940.0 ± ∞ ¹     871.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.95Ki ± ∞ ¹   11.87Ki ± ∞ ¹  ~ (p=0.500 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.912Ki ± ∞ ¹   8.843Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.5Ki ± ∞ ¹   118.2Ki ± ∞ ¹  ~ (p=0.600 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     444.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.468Ki ± ∞ ¹   2.235Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.688Ki ± ∞ ¹   3.561Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   369.0 ± ∞ ¹     363.0 ± ∞ ¹  ~ (p=0.700 n=3) ²
geomean                                                  2.113Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                37.00 ± ∞ ¹   30.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.252                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.558 ± ∞ ¹     3.882 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.104 ± ∞ ¹     4.744 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.812 ± ∞ ¹     4.840 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.054 ± ∞ ¹     5.102 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                    4.584           4.618        +0.73%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.083 ± ∞ ¹   5.541 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.247 ± ∞ ¹   7.695 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.751 ± ∞ ¹   7.801 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.142 ± ∞ ¹   8.226 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                  7.172         7.232        +0.85%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   700.4m ± ∞ ¹        699.9m ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        616.5m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.9m ± ∞ ¹        620.2m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.5m ± ∞ ¹        620.2m ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                       639.3m              638.3m        -0.16%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.22k ± ∞ ¹         14.41k ± ∞ ¹  ~ (p=0.700 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T05:53:33Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │               new.bench               │
                                                       │    sec/op    │    sec/op      vs base                │
Write_SingleEntry-4                                      822.6n ± ∞ ¹    934.2n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       2.010µ ± ∞ ¹    2.083µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                       25.50µ ± ∞ ¹    25.15µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                        278.2µ ± ∞ ¹    287.7µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                        2.909m ± ∞ ¹    2.929m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              81.61µ ± ∞ ¹    81.86µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.474µ ± ∞ ¹    1.564µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.11µ ± ∞ ¹    12.18µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      17.67µ ± ∞ ¹    16.50µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    116.2µ ± ∞ ¹    115.5µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     185.9µ ± ∞ ¹    174.6µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.223m ± ∞ ¹    1.222m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4                                       949.2n ± ∞ ¹   1024.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.036µ ± ∞ ¹    1.130µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                1.600µ ± ∞ ¹    1.686µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              3.048µ ± ∞ ¹    3.030µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Consumer_SequentialRead-4                                151.3µ ± ∞ ¹    169.9µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  150.2µ ± ∞ ¹    199.1µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                256.1n ± ∞ ¹    236.3n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.407µ ± ∞ ¹    3.318µ ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  16.20µ          17.01µ        +4.96%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.82 ± ∞ ¹   14.87 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4             44.43 ± ∞ ¹   49.71 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_LargeBatch-4             59.13 ± ∞ ¹   67.39 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4              59.05 ± ∞ ¹   61.25 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4              55.43 ± ∞ ¹   77.82 ± ∞ ¹       ~ (p=0.500 n=3) ²
Write_Concurrent-4             8.883 ± ∞ ¹   8.399 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentMultiShard-4   9.023 ± ∞ ¹   8.087 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      29.93 ± ∞ ¹   23.45 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4    2.897 ± ∞ ¹   2.880 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                        21.56         21.75        +0.90%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         145.0k ± ∞ ¹      128.2k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                          504.9k ± ∞ ¹      564.8k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_LargeBatch-4                                          604.6k ± ∞ ¹      689.1k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                           564.0k ± ∞ ¹      585.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                           520.0k ± ∞ ¹      730.0k ± ∞ ¹       ~ (p=0.500 n=3) ²
Write_CompressibleEntries-4                                 14.10k ± ∞ ¹      14.30k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          83.70k ± ∞ ¹      76.78k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.591k ± ∞ ¹      9.297k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         82.84k ± ∞ ¹     114.06k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       10.32k ± ∞ ¹      10.28k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        115.4k ± ∞ ¹      102.4k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.600k ± ∞ ¹      9.400k ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_Concurrent-4                                          109.7k ± ∞ ¹      103.7k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentMultiShard-4                               108.71k ± ∞ ¹      97.44k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   383.7k ± ∞ ¹      300.6k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                 36.25k ± ∞ ¹      36.04k ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                     91.24k            92.11k        +0.95%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         948.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.729Ki ± ∞ ¹   8.656Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.31Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.5Ki ± ∞ ¹   835.0Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              6.514Ki ± ∞ ¹   6.427Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        161.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.255Ki ± ∞ ¹   1.190Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        940.0 ± ∞ ¹     870.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.76Ki ± ∞ ¹   11.70Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.909Ki ± ∞ ¹   8.866Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.6Ki ± ∞ ¹   118.9Ki ± ∞ ¹  ~ (p=0.600 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  515.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.588Ki ± ∞ ¹   2.446Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Consumer_RandomAccess-4                                  3.722Ki ± ∞ ¹   3.633Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   423.0 ± ∞ ¹     463.0 ± ∞ ¹  ~ (p=0.700 n=3) ²
geomean                                                  2.130Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                38.00 ± ∞ ¹   32.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.264                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.803 ± ∞ ¹     3.858 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       4.968 ± ∞ ¹     4.825 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      5.359 ± ∞ ¹     5.336 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     4.998 ± ∞ ¹     4.901 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                    4.743           4.697        -0.96%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.421 ± ∞ ¹   5.498 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.028 ± ∞ ¹   7.782 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    8.638 ± ∞ ¹   8.604 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.058 ± ∞ ¹   7.891 ± ∞ ¹       ~ (p=0.600 n=3) ²
geomean                                                  7.419         7.341        -1.04%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   703.3m ± ∞ ¹        702.5m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          620.1m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.4m ± ∞ ¹        620.7m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.4m ± ∞ ¹        621.1m ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                                                       640.1m              640.2m        +0.01%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         14.10k ± ∞ ¹         14.30k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T14:25:02Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │  old.bench   │               new.bench               │
                                                       │    sec/op    │    sec/op      vs base                │
Write_SingleEntry-4                                      828.1n ± ∞ ¹    916.4n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       2.009µ ± ∞ ¹    2.090µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       28.53µ ± ∞ ¹    21.51µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_HugeBatch-4                                        291.7µ ± ∞ ¹    331.2µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                        3.482m ± ∞ ¹    3.014m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              82.19µ ± ∞ ¹    81.47µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       1.428µ ± ∞ ¹    1.528µ ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     12.17µ ± ∞ ¹    12.06µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      19.78µ ± ∞ ¹    17.36µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    117.0µ ± ∞ ¹    115.4µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     183.4µ ± ∞ ¹    163.0µ ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   1.210m ± ∞ ¹    1.258m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       958.6n ± ∞ ¹   1015.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             1.070µ ± ∞ ¹    1.107µ ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_ConcurrentBatched-4                                1.596µ ± ∞ ¹    1.675µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentReadWrite-4                              2.917µ ± ∞ ¹    3.007µ ± ∞ ¹       ~ (p=0.700 n=3) ²
Consumer_SequentialRead-4                                180.9µ ± ∞ ¹    151.0µ ± ∞ ¹       ~ (p=0.200 n=3) ²
Consumer_RandomAccess-4                                  174.8µ ± ∞ ¹    141.9µ ± ∞ ¹       ~ (p=0.400 n=3) ²
Consumer_LagMeasurement-4                                256.5n ± ∞ ¹    237.0n ± ∞ ¹       ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 2.349µ ± ∞ ¹    2.844µ ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                                                  16.80µ          16.44µ        -2.17%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            14.74 ± ∞ ¹   14.34 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_SmallBatch-4             45.13 ± ∞ ¹   50.36 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_LargeBatch-4             68.23 ± ∞ ¹   78.73 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4              54.34 ± ∞ ¹   34.76 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_MegaBatch-4              53.30 ± ∞ ¹   59.70 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_Concurrent-4             8.927 ± ∞ ¹   8.539 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.097 ± ∞ ¹   8.590 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      29.39 ± ∞ ¹   28.09 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentReadWrite-4    2.868 ± ∞ ¹   2.879 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                        21.29         20.71        -2.75%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         127.1k ± ∞ ¹      123.7k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_SmallBatch-4                                          512.9k ± ∞ ¹      572.2k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_LargeBatch-4                                          697.6k ± ∞ ¹      805.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                           519.0k ± ∞ ¹      332.0k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_MegaBatch-4                                           500.0k ± ∞ ¹      560.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                                 14.61k ± ∞ ¹      14.54k ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          71.99k ± ∞ ¹      68.03k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.634k ± ∞ ¹      9.630k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         107.7k ± ∞ ¹      125.6k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.120k ± ∞ ¹     10.330k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        114.0k ± ∞ ¹      104.6k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.600k ± ∞ ¹      9.200k ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_Concurrent-4                                          110.2k ± ∞ ¹      105.4k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                109.6k ± ∞ ¹      103.5k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   376.8k ± ∞ ¹      360.1k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentReadWrite-4                                 35.89k ± ∞ ¹      36.03k ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                     90.71k            89.78k        -1.02%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         948.0 ± ∞ ¹     875.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.727Ki ± ∞ ¹   8.655Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.40Ki ± ∞ ¹   84.39Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.1Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              6.473Ki ± ∞ ¹   6.407Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.254Ki ± ∞ ¹   1.184Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        942.0 ± ∞ ¹     869.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.98Ki ± ∞ ¹   11.68Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.915Ki ± ∞ ¹   8.856Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.6Ki ± ∞ ¹   119.3Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  515.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                2.750Ki ± ∞ ¹   2.344Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.725Ki ± ∞ ¹   3.565Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   407.0 ± ∞ ¹     405.0 ± ∞ ¹  ~ (p=0.700 n=3) ²
geomean                                                  2.135Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench  │            new.bench             │
                                                       │  allocs/op  │  allocs/op   vs base             │
Write_SingleEntry-4                                      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                              17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                             7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                              8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                39.00 ± ∞ ¹   31.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  45.00 ± ∞ ¹   41.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                 5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  9.276                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.940 ± ∞ ¹     3.939 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.010 ± ∞ ¹     4.998 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.741 ± ∞ ¹     5.364 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.001 ± ∞ ¹     4.794 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                    4.651           4.743        +1.98%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.618 ± ∞ ¹   5.591 ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.064 ± ∞ ¹   8.060 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.633 ± ∞ ¹   8.646 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.058 ± ∞ ¹   7.723 ± ∞ ¹       ~ (p=0.800 n=3) ²
geomean                                                  7.265         7.406        +1.94%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   702.7m ± ∞ ¹        703.5m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         621.1m ± ∞ ¹        620.7m ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.6m ± ∞ ¹        620.8m ± ∞ ¹       ~ (p=0.300 n=3) ²
geomean                                                       639.9m              640.3m        +0.07%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         14.61k ± ∞ ¹         14.54k ± ∞ ¹  ~ (p=0.600 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T17:21:32Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │   old.bench    │               new.bench                │
                                                       │     sec/op     │    sec/op      vs base                 │
Write_SingleEntry-4                                        821.7n ± ∞ ¹    915.3n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         2.020µ ± ∞ ¹    2.032µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                         23.86µ ± ∞ ¹    24.90µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                          304.1µ ± ∞ ¹    275.1µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                          2.820m ± ∞ ¹    2.439m ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                                81.20µ ± ∞ ¹    81.33µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Single-4         1.466µ ± ∞ ¹    2.122µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       12.02µ ± ∞ ¹    11.85µ ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        18.06µ ± ∞ ¹    16.11µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      113.6µ ± ∞ ¹    114.4µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       190.3µ ± ∞ ¹    159.0µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     1.203m ± ∞ ¹    1.195m ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_Concurrent-4                                         943.7n ± ∞ ¹   1008.0n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               1.016µ ± ∞ ¹    1.064µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  1.556µ ± ∞ ¹    1.585µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                2.914µ ± ∞ ¹    2.926µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Consumer_SequentialRead-4                                 151.83µ ± ∞ ¹    11.66µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  134.534µ ± ∞ ¹    6.561µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  253.3n ± ∞ ¹    237.5n ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   2.156µ ± ∞ ¹    2.689µ ± ∞ ¹        ~ (p=0.100 n=3) ²
geomean                                                    15.93µ          12.21µ        -23.36%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.91 ± ∞ ¹   13.32 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_SmallBatch-4             45.51 ± ∞ ¹   43.47 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4             77.19 ± ∞ ¹   59.74 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              51.51 ± ∞ ¹   57.16 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_MegaBatch-4              47.97 ± ∞ ¹   58.63 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4             9.340 ± ∞ ¹   8.774 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.452 ± ∞ ¹   8.940 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.85 ± ∞ ¹   29.21 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4    3.019 ± ∞ ¹   2.975 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                        21.57         20.99        -2.70%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         145.8k ± ∞ ¹      114.8k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_SmallBatch-4                                          517.2k ± ∞ ¹      493.9k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                          789.3k ± ∞ ¹      610.8k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           492.0k ± ∞ ¹      546.0k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_MegaBatch-4                                           450.0k ± ∞ ¹      550.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                                 14.72k ± ∞ ¹      14.50k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          73.50k ± ∞ ¹      67.97k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.907k ± ∞ ¹      9.873k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        102.87k ± ∞ ¹      87.30k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.920k ± ∞ ¹      9.140k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        111.1k ± ∞ ¹      109.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.700k ± ∞ ¹      9.700k ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_Concurrent-4                                          115.3k ± ∞ ¹      108.3k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                113.9k ± ∞ ¹      107.7k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   331.4k ± ∞ ¹      374.5k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                 37.77k ± ∞ ¹      37.23k ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                     91.82k            88.40k        -3.73%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     872.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.728Ki ± ∞ ¹   8.657Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.5Ki ± ∞ ¹   835.3Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                              6.458Ki ± ∞ ¹   6.411Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.249Ki ± ∞ ¹   1.179Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        944.0 ± ∞ ¹     870.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.82Ki ± ∞ ¹   11.90Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.917Ki ± ∞ ¹   8.845Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.5Ki ± ∞ ¹   118.4Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                 2504.0 ± ∞ ¹     115.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.700Ki ± ∞ ¹   2.099Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   387.0 ± ∞ ¹     382.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
geomean                                                  2.114Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench   │            new.bench             │
                                                       │  allocs/op   │  allocs/op   vs base             │
Write_SingleEntry-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                         6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                         9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_CompressibleEntries-4                               17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4      8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4     17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4    108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                              7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                 6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                               8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                36.000 ± ∞ ¹   1.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                   45.00 ± ∞ ¹   18.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                 2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                  5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                   9.239                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.969 ± ∞ ¹     3.915 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.132 ± ∞ ¹     5.124 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      5.152 ± ∞ ¹     4.754 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.052 ± ∞ ¹     5.056 ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                                                    4.798           4.686        -2.34%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.660 ± ∞ ¹   5.575 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.292 ± ∞ ¹   8.264 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    8.303 ± ∞ ¹   7.650 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.142 ± ∞ ¹   8.142 ± ∞ ¹       ~ (p=0.600 n=3) ²
geomean                                                  7.505         7.319        -2.48%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   703.0m ± ∞ ¹        702.0m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         621.0m ± ∞ ¹        621.3m ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        621.4m ± ∞ ¹        620.9m ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                       640.1m              640.1m        +0.00%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         14.72k ± ∞ ¹         14.50k ± ∞ ¹  ~ (p=0.400 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T18:52:56Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │   old.bench    │               new.bench                │
                                                       │     sec/op     │    sec/op      vs base                 │
Write_SingleEntry-4                                        802.9n ± ∞ ¹    872.3n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         1.967µ ± ∞ ¹    2.184µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                         24.44µ ± ∞ ¹    28.75µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                          301.6µ ± ∞ ¹    293.0µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                          2.751m ± ∞ ¹    3.086m ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                                80.73µ ± ∞ ¹    79.29µ ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Single-4         1.466µ ± ∞ ¹    1.491µ ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       11.97µ ± ∞ ¹    11.86µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        19.85µ ± ∞ ¹    17.70µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      115.0µ ± ∞ ¹    113.4µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       176.4µ ± ∞ ¹    179.6µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     1.189m ± ∞ ¹    1.175m ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_Concurrent-4                                         930.1n ± ∞ ¹    979.0n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               997.5n ± ∞ ¹   1076.0n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  1.543µ ± ∞ ¹    1.580µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                2.855µ ± ∞ ¹    2.865µ ± ∞ ¹        ~ (p=0.300 n=3) ²
Consumer_SequentialRead-4                                 137.82µ ± ∞ ¹    11.54µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  129.124µ ± ∞ ¹    6.625µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  251.7n ± ∞ ¹    261.1n ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   2.180µ ± ∞ ¹    2.751µ ± ∞ ¹        ~ (p=0.100 n=3) ²
geomean                                                    15.74µ          12.43µ        -21.03%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.42 ± ∞ ¹   15.72 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_SmallBatch-4             47.11 ± ∞ ¹   51.21 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4             59.94 ± ∞ ¹   77.85 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4              68.05 ± ∞ ¹   72.66 ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_MegaBatch-4              51.17 ± ∞ ¹   61.83 ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_Concurrent-4             9.465 ± ∞ ¹   9.817 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.805 ± ∞ ¹   8.477 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      26.51 ± ∞ ¹   29.67 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4    3.003 ± ∞ ¹   2.996 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                        21.97         23.37        +6.39%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         141.5k ± ∞ ¹      135.5k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_SmallBatch-4                                          535.3k ± ∞ ¹      581.9k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                          612.9k ± ∞ ¹      796.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                           650.0k ± ∞ ¹      694.0k ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_MegaBatch-4                                           480.0k ± ∞ ¹      580.0k ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_CompressibleEntries-4                                 13.03k ± ∞ ¹      15.18k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          72.77k ± ∞ ¹      80.56k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.760k ± ∞ ¹      9.626k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         124.0k ± ∞ ¹      123.2k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.670k ± ∞ ¹     10.440k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4        121.3k ± ∞ ¹      102.9k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.800k ± ∞ ¹      9.800k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4                                          116.8k ± ∞ ¹      121.2k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                118.1k ± ∞ ¹      102.1k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   339.8k ± ∞ ¹      380.4k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                 37.58k ± ∞ ¹      37.48k ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                     93.43k            97.63k        +4.50%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.729Ki ± ∞ ¹   8.656Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.39Ki ± ∞ ¹   84.32Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.2Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              6.606Ki ± ∞ ¹   6.359Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.252Ki ± ∞ ¹   1.184Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        940.0 ± ∞ ¹     869.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.87Ki ± ∞ ¹   11.68Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.909Ki ± ∞ ¹   8.842Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.3Ki ± ∞ ¹   118.2Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               187.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                 2489.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.702Ki ± ∞ ¹   2.099Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   394.0 ± ∞ ¹     392.0 ± ∞ ¹  ~ (p=0.700 n=3) ²
geomean                                                  2.117Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench   │            new.bench             │
                                                       │  allocs/op   │  allocs/op   vs base             │
Write_SingleEntry-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                         6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                         9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                               17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4      8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4     17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4    108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                              7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                 6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                               8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                36.000 ± ∞ ¹   1.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                   45.00 ± ∞ ¹   18.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                 2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                  5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                   9.239                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.522 ± ∞ ¹     4.088 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.065 ± ∞ ¹     4.986 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      5.031 ± ∞ ¹     5.415 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.105 ± ∞ ¹     5.109 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                    4.627           4.873        +5.33%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.010 ± ∞ ¹   5.837 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.169 ± ∞ ¹   8.057 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    8.094 ± ∞ ¹   8.738 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.226 ± ∞ ¹   8.226 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                  7.225         7.625        +5.54%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   703.0m ± ∞ ¹        700.7m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          620.1m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.5m ± ∞ ¹        620.4m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.3m ± ∞ ¹        621.0m ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                                                       640.0m              639.3m        -0.11%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.03k ± ∞ ¹         15.18k ± ∞ ¹  ~ (p=0.100 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T18:56:23Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │   old.bench    │               new.bench               │
                                                       │     sec/op     │    sec/op     vs base                 │
Write_SingleEntry-4                                        821.2n ± ∞ ¹   882.3n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         2.050µ ± ∞ ¹   2.068µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_LargeBatch-4                                         25.67µ ± ∞ ¹   22.95µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_HugeBatch-4                                          317.0µ ± ∞ ¹   264.5µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                          2.895m ± ∞ ¹   2.694m ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                                81.21µ ± ∞ ¹   81.31µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Single-4         1.524µ ± ∞ ¹   1.526µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       12.25µ ± ∞ ¹   12.01µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        19.26µ ± ∞ ¹   15.54µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      116.6µ ± ∞ ¹   114.9µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       169.4µ ± ∞ ¹   180.4µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     1.193m ± ∞ ¹   1.218m ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_Concurrent-4                                         943.0n ± ∞ ¹   996.8n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               1.023µ ± ∞ ¹   1.100µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  1.587µ ± ∞ ¹   1.593µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_ConcurrentReadWrite-4                                2.896µ ± ∞ ¹   3.026µ ± ∞ ¹        ~ (p=0.200 n=3) ²
Consumer_SequentialRead-4                                 158.39µ ± ∞ ¹   11.52µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  157.342µ ± ∞ ¹   6.724µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  254.8n ± ∞ ¹   257.6n ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   2.134µ ± ∞ ¹   2.607µ ± ∞ ¹        ~ (p=0.100 n=3) ²
geomean                                                    16.25µ         12.13µ        -25.30%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            14.39 ± ∞ ¹   15.49 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_SmallBatch-4             44.25 ± ∞ ¹   50.93 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4             47.06 ± ∞ ¹   72.71 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_HugeBatch-4              73.70 ± ∞ ¹   63.76 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4              54.37 ± ∞ ¹   54.37 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4             8.904 ± ∞ ¹   9.402 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.300 ± ∞ ¹   8.633 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.36 ± ∞ ¹   28.66 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4    2.972 ± ∞ ¹   2.952 ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                        20.86         22.30        +6.89%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         124.0k ± ∞ ¹      133.5k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_SmallBatch-4                                          502.8k ± ∞ ¹      578.8k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                          481.2k ± ∞ ¹      743.5k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                           704.0k ± ∞ ¹      609.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                           510.0k ± ∞ ¹      510.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                                 14.66k ± ∞ ¹      14.70k ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          79.78k ± ∞ ¹      74.31k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.386k ± ∞ ¹      9.451k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         124.7k ± ∞ ¹      118.3k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.550k ± ∞ ¹     10.210k ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       112.30k ± ∞ ¹      83.50k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.800k ± ∞ ¹      9.100k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                          109.9k ± ∞ ¹      116.1k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                112.0k ± ∞ ¹      104.0k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   325.1k ± ∞ ¹      367.5k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                 37.19k ± ∞ ¹      36.94k ± ∞ ¹       ~ (p=0.200 n=3) ²
geomean                                                     91.24k            92.29k        +1.15%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         948.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.728Ki ± ∞ ¹   8.654Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.34Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.1Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              6.468Ki ± ∞ ¹   6.391Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.259Ki ± ∞ ¹   1.188Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        940.0 ± ∞ ¹     870.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.89Ki ± ∞ ¹   11.71Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.907Ki ± ∞ ¹   8.823Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.3Ki ± ∞ ¹   119.5Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                 2724.0 ± ∞ ¹     119.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.717Ki ± ∞ ¹   2.099Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   385.0 ± ∞ ¹     368.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  2.124Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench   │            new.bench             │
                                                       │  allocs/op   │  allocs/op   vs base             │
Write_SingleEntry-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                         6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                         9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                               17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4      8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4     17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4    108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                              7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                 6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                               8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                38.000 ± ∞ ¹   1.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                   45.00 ± ∞ ¹   18.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                 2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                  5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                   9.264                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.960 ± ∞ ¹     3.978 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       4.862 ± ∞ ¹     4.896 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.955 ± ∞ ¹     5.299 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.101 ± ∞ ¹     4.748 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                    4.697           4.705        +0.17%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.637 ± ∞ ¹   5.652 ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     7.856 ± ∞ ¹   7.910 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.993 ± ∞ ¹   8.546 ± ∞ ¹       ~ (p=0.300 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.226 ± ∞ ¹   7.639 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  7.346         7.350        +0.06%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   702.5m ± ∞ ¹        701.7m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        618.9m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.7m ± ∞ ¹        621.1m ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.5m ± ∞ ¹        621.6m ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                       639.7m              639.9m        +0.03%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         14.66k ± ∞ ¹         14.70k ± ∞ ¹  ~ (p=0.800 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T19:04:06Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │   old.bench    │               new.bench               │
                                                       │     sec/op     │    sec/op     vs base                 │
Write_SingleEntry-4                                        813.9n ± ∞ ¹   871.9n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         2.077µ ± ∞ ¹   2.061µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_LargeBatch-4                                         27.02µ ± ∞ ¹   24.79µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_HugeBatch-4                                          315.3µ ± ∞ ¹   286.4µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                          2.655m ± ∞ ¹   3.096m ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                                80.79µ ± ∞ ¹   78.88µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4         1.454µ ± ∞ ¹   1.500µ ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       12.12µ ± ∞ ¹   11.78µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        15.51µ ± ∞ ¹   18.75µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      116.8µ ± ∞ ¹   115.0µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       181.2µ ± ∞ ¹   176.3µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     1.198m ± ∞ ¹   1.196m ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_Concurrent-4                                         962.1n ± ∞ ¹   994.8n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               1.056µ ± ∞ ¹   1.123µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  1.607µ ± ∞ ¹   1.630µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                3.109µ ± ∞ ¹   3.023µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                 141.87µ ± ∞ ¹   11.54µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  136.193µ ± ∞ ¹   6.713µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  259.2n ± ∞ ¹   262.7n ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   2.499µ ± ∞ ¹   2.857µ ± ∞ ¹        ~ (p=0.700 n=3) ²
geomean                                                    16.09µ         12.45µ        -22.64%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            14.36 ± ∞ ¹   15.44 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_SmallBatch-4             45.05 ± ∞ ¹   50.02 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4             79.19 ± ∞ ¹   75.43 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              60.30 ± ∞ ¹   72.24 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              54.37 ± ∞ ¹   59.70 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4             9.051 ± ∞ ¹   9.542 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.322 ± ∞ ¹   7.604 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      24.83 ± ∞ ¹   28.93 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4    2.901 ± ∞ ¹   2.783 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                        21.59         22.48        +4.12%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         123.8k ± ∞ ¹      133.1k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_SmallBatch-4                                          511.9k ± ∞ ¹      568.4k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                          809.7k ± ∞ ¹      771.3k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           576.0k ± ∞ ¹      690.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           510.0k ± ∞ ¹      560.0k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                                 14.64k ± ∞ ¹      15.06k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          70.64k ± ∞ ¹      78.28k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.667k ± ∞ ¹      9.470k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         87.55k ± ∞ ¹     124.75k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.400k ± ∞ ¹     10.380k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       112.00k ± ∞ ¹      92.80k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.700k ± ∞ ¹      9.400k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_Concurrent-4                                          111.7k ± ∞ ¹      117.8k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               112.32k ± ∞ ¹      91.61k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   318.3k ± ∞ ¹      370.9k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                 36.30k ± ∞ ¹      34.83k ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                     90.30k            94.38k        +4.52%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.727Ki ± ∞ ¹   8.653Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.38Ki ± ∞ ¹   84.31Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.1Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              6.470Ki ± ∞ ¹   6.368Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.253Ki ± ∞ ¹   1.187Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        943.0 ± ∞ ¹     869.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.92Ki ± ∞ ¹   11.68Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.922Ki ± ∞ ¹   8.844Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.5Ki ± ∞ ¹   119.0Ki ± ∞ ¹  ~ (p=1.000 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                 2534.0 ± ∞ ¹     117.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.700Ki ± ∞ ¹   2.099Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   435.0 ± ∞ ¹     406.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  2.129Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench   │            new.bench             │
                                                       │  allocs/op   │  allocs/op   vs base             │
Write_SingleEntry-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                         6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                         9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                               17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4      8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4     17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4    108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                              7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                 6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                               8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                37.000 ± ∞ ¹   1.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                   45.00 ± ∞ ¹   18.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                 2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                  5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                   9.252                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.962 ± ∞ ¹     4.054 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.008 ± ∞ ¹     4.896 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.880 ± ∞ ¹     5.392 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.047 ± ∞ ¹     4.898 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                    4.702           4.785        +1.77%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.629 ± ∞ ¹   5.791 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.091 ± ∞ ¹   7.926 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.868 ± ∞ ¹   8.688 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.142 ± ∞ ¹   7.891 ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                  7.349         7.490        +1.91%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   703.8m ± ∞ ¹        702.0m ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          621.3m ± ∞ ¹        617.7m ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.4m ± ∞ ¹        620.7m ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.2m ± ∞ ¹        620.6m ± ∞ ¹       ~ (p=0.800 n=3) ²
geomean                                                       640.5m              639.3m        -0.18%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         14.64k ± ∞ ¹         15.06k ± ∞ ¹  ~ (p=0.400 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T19:07:24Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │   old.bench    │               new.bench                │
                                                       │     sec/op     │    sec/op      vs base                 │
Write_SingleEntry-4                                        820.5n ± ∞ ¹    901.1n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         2.025µ ± ∞ ¹    2.086µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_LargeBatch-4                                         22.06µ ± ∞ ¹    23.34µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                          258.8µ ± ∞ ¹    277.7µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_MegaBatch-4                                          3.270m ± ∞ ¹    2.491m ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                                81.15µ ± ∞ ¹    80.10µ ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Single-4         1.461µ ± ∞ ¹    1.550µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       12.14µ ± ∞ ¹    11.98µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        13.52µ ± ∞ ¹    15.36µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      115.8µ ± ∞ ¹    116.0µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       168.6µ ± ∞ ¹    201.9µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     1.195m ± ∞ ¹    1.213m ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_Concurrent-4                                         942.1n ± ∞ ¹   1021.0n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               1.020µ ± ∞ ¹    1.111µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  1.596µ ± ∞ ¹    1.640µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_ConcurrentReadWrite-4                                2.980µ ± ∞ ¹    3.040µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Consumer_SequentialRead-4                                 162.81µ ± ∞ ¹    11.62µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  155.498µ ± ∞ ¹    6.913µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  255.0n ± ∞ ¹    261.9n ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   2.036µ ± ∞ ¹    2.967µ ± ∞ ¹        ~ (p=0.100 n=3) ²
geomean                                                    15.72µ          12.36µ        -21.34%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            14.53 ± ∞ ¹   14.90 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4             45.59 ± ∞ ¹   50.01 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4             74.90 ± ∞ ¹   76.49 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4              64.91 ± ∞ ¹   63.03 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              63.96 ± ∞ ¹   52.23 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_Concurrent-4             9.185 ± ∞ ¹   9.412 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.368 ± ∞ ¹   8.788 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      25.94 ± ∞ ¹   28.56 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_ConcurrentReadWrite-4    2.938 ± ∞ ¹   2.923 ± ∞ ¹       ~ (p=0.800 n=3) ²
geomean                        22.28         22.17        -0.46%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         125.3k ± ∞ ¹      128.4k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4                                          518.1k ± ∞ ¹      568.3k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_LargeBatch-4                                          765.8k ± ∞ ¹      782.1k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                           620.0k ± ∞ ¹      602.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           600.0k ± ∞ ¹      490.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                                 13.17k ± ∞ ¹      14.79k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          70.99k ± ∞ ¹      76.89k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.643k ± ∞ ¹      9.212k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         108.3k ± ∞ ¹      120.5k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.280k ± ∞ ¹     10.270k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       119.00k ± ∞ ¹      93.20k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.800k ± ∞ ¹      9.300k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4                                          113.4k ± ∞ ¹      116.2k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                                112.9k ± ∞ ¹      105.9k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   332.6k ± ∞ ¹      366.2k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_ConcurrentReadWrite-4                                 36.77k ± ∞ ¹      36.57k ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                     92.86k            92.97k        +0.12%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         948.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.726Ki ± ∞ ¹   8.653Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.39Ki ± ∞ ¹   84.34Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.2Ki ± ∞ ¹   835.2Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_CompressibleEntries-4                              6.598Ki ± ∞ ¹   6.384Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.254Ki ± ∞ ¹   1.192Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        942.0 ± ∞ ¹     869.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.95Ki ± ∞ ¹   11.70Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.909Ki ± ∞ ¹   8.842Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.3Ki ± ∞ ¹   119.2Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                 2632.0 ± ∞ ¹     119.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.708Ki ± ∞ ¹   2.099Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   373.0 ± ∞ ¹     413.0 ± ∞ ¹  ~ (p=0.300 n=3) ²
geomean                                                  2.119Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench   │            new.bench             │
                                                       │  allocs/op   │  allocs/op   vs base             │
Write_SingleEntry-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                         6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                         9.000 ± ∞ ¹   8.000 ± ∞ ¹  ~ (p=0.300 n=3) ²
Write_CompressibleEntries-4                               17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4      8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4     17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4    108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                              7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                 6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                               8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                38.000 ± ∞ ¹   1.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                   45.00 ± ∞ ¹   18.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                 2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                  5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                   9.264                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.553 ± ∞ ¹     3.998 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       5.005 ± ∞ ¹     4.790 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.820 ± ∞ ¹     5.330 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.101 ± ∞ ¹     4.846 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                    4.573           4.716        +3.13%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.064 ± ∞ ¹   5.687 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.071 ± ∞ ¹   7.710 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.767 ± ∞ ¹   8.596 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.226 ± ∞ ¹   7.807 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                  7.149         7.365        +3.03%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   701.7m ± ∞ ¹        702.0m ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          620.1m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.5m ± ∞ ¹        620.2m ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.1m ± ∞ ¹        620.7m ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                       639.7m              639.8m        +0.02%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.17k ± ∞ ¹         14.79k ± ∞ ¹  ~ (p=0.100 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T19:41:27Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │   old.bench    │               new.bench                │
                                                       │     sec/op     │    sec/op      vs base                 │
Write_SingleEntry-4                                        840.1n ± ∞ ¹    915.1n ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_SmallBatch-4                                         1.959µ ± ∞ ¹    1.996µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                         21.02µ ± ∞ ¹    26.51µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_HugeBatch-4                                          282.0µ ± ∞ ¹    300.7µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                          2.787m ± ∞ ¹    3.030m ± ∞ ¹        ~ (p=0.200 n=3) ²
Write_CompressibleEntries-4                                80.77µ ± ∞ ¹    79.81µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Single-4         1.493µ ± ∞ ¹    1.525µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       12.02µ ± ∞ ¹    11.70µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        20.32µ ± ∞ ¹    16.02µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      113.7µ ± ∞ ¹    116.2µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       169.0µ ± ∞ ¹    166.7µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     1.186m ± ∞ ¹    1.179m ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_Concurrent-4                                         938.2n ± ∞ ¹   1012.0n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               1.001µ ± ∞ ¹    1.101µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  1.526µ ± ∞ ¹    1.617µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                2.894µ ± ∞ ¹    2.905µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Consumer_SequentialRead-4                                 157.40µ ± ∞ ¹    11.63µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  132.148µ ± ∞ ¹    6.742µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  253.2n ± ∞ ¹    261.9n ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   2.281µ ± ∞ ¹    2.584µ ± ∞ ¹        ~ (p=0.100 n=3) ²
geomean                                                    15.77µ          12.31µ        -21.96%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.86 ± ∞ ¹   13.92 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_SmallBatch-4             46.82 ± ∞ ¹   51.80 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4             52.42 ± ∞ ¹   75.93 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              68.89 ± ∞ ¹   60.72 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4              55.43 ± ∞ ¹   59.70 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_Concurrent-4             9.321 ± ∞ ¹   9.488 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentMultiShard-4   9.688 ± ∞ ¹   7.952 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      26.08 ± ∞ ¹   28.90 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_ConcurrentReadWrite-4    3.058 ± ∞ ¹   3.044 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                        21.86         22.21        +1.64%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         145.4k ± ∞ ¹      120.0k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_SmallBatch-4                                          532.0k ± ∞ ¹      588.6k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                          536.0k ± ∞ ¹      776.4k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           658.0k ± ∞ ¹      580.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                           520.0k ± ∞ ¹      560.0k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                                 13.32k ± ∞ ¹      14.84k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          71.39k ± ∞ ¹      78.34k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.632k ± ∞ ¹      9.441k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         131.0k ± ∞ ¹      122.9k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.490k ± ∞ ¹     10.350k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       105.80k ± ∞ ¹      97.50k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.700k ± ∞ ¹      9.600k ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_Concurrent-4                                          115.1k ± ∞ ¹      117.1k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_ConcurrentMultiShard-4                               116.72k ± ∞ ¹      95.81k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   334.3k ± ∞ ¹      370.5k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_ConcurrentReadWrite-4                                 38.27k ± ∞ ¹      38.09k ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                     92.45k            93.95k        +1.63%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.729Ki ± ∞ ¹   8.654Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.40Ki ± ∞ ¹   84.35Ki ± ∞ ¹  ~ (p=0.200 n=3) ²
Write_MegaBatch-4                                        835.4Ki ± ∞ ¹   835.2Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              6.584Ki ± ∞ ¹   6.385Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.253Ki ± ∞ ¹   1.188Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        943.0 ± ∞ ¹     870.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.90Ki ± ∞ ¹   11.68Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.933Ki ± ∞ ¹   8.826Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   118.4Ki ± ∞ ¹   118.6Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  516.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                 2652.0 ± ∞ ¹     118.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.701Ki ± ∞ ¹   2.099Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   407.0 ± ∞ ¹     367.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
geomean                                                  2.129Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench   │            new.bench             │
                                                       │  allocs/op   │  allocs/op   vs base             │
Write_SingleEntry-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                         6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                         9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                               17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4      8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4     17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4    108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                              7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                 6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                               8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                38.000 ± ∞ ¹   1.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                   45.00 ± ∞ ¹   18.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                 2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                  5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                   9.264                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.604 ± ∞ ¹     4.008 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       4.989 ± ∞ ¹     4.881 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.933 ± ∞ ¹     5.373 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     5.055 ± ∞ ¹     5.001 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                    4.602           4.788        +4.06%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.122 ± ∞ ¹   5.706 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     8.062 ± ∞ ¹   7.902 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.943 ± ∞ ¹   8.663 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   8.142 ± ∞ ¹   8.058 ± ∞ ¹       ~ (p=0.600 n=3) ²
geomean                                                  7.189         7.490        +4.19%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   703.5m ± ∞ ¹        702.5m ± ∞ ¹       ~ (p=0.600 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        617.7m ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         621.0m ± ∞ ¹        620.3m ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.8m ± ∞ ¹        620.8m ± ∞ ¹       ~ (p=1.000 n=3) ²
geomean                                                       640.1m              639.4m        -0.11%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.32k ± ∞ ¹         14.84k ± ∞ ¹  ~ (p=0.100 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

github-actions · 2025-08-07T19:47:59Z

Comet Benchmark Results

✅ No significant performance regression detected

Benchmark comparison

goos: linux
goarch: amd64
pkg: github.com/orbiterhq/comet
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │   old.bench    │               new.bench                │
                                                       │     sec/op     │    sec/op      vs base                 │
Write_SingleEntry-4                                        812.6n ± ∞ ¹    915.1n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         1.987µ ± ∞ ¹    2.061µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                         22.98µ ± ∞ ¹    23.04µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_HugeBatch-4                                          299.6µ ± ∞ ¹    288.7µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_MegaBatch-4                                          2.927m ± ∞ ¹    2.959m ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressibleEntries-4                                82.03µ ± ∞ ¹    80.24µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4         1.540µ ± ∞ ¹    1.530µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       12.33µ ± ∞ ¹    12.21µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        16.32µ ± ∞ ¹    16.46µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      117.0µ ± ∞ ¹    115.0µ ± ∞ ¹        ~ (p=0.700 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       124.8µ ± ∞ ¹    190.3µ ± ∞ ¹        ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     1.215m ± ∞ ¹    1.212m ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_Concurrent-4                                         970.0n ± ∞ ¹   1022.0n ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               1.053µ ± ∞ ¹    1.131µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  1.654µ ± ∞ ¹    1.655µ ± ∞ ¹        ~ (p=1.000 n=3) ²
Write_ConcurrentReadWrite-4                                3.030µ ± ∞ ¹    2.965µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                 169.81µ ± ∞ ¹    11.86µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  168.047µ ± ∞ ¹    6.827µ ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  263.0n ± ∞ ¹    256.9n ± ∞ ¹        ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   2.677µ ± ∞ ¹    2.635µ ± ∞ ¹        ~ (p=1.000 n=3) ²
geomean                                                    16.18µ          12.41µ        -23.27%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                             │  old.bench  │              new.bench              │
                             │ MB_written  │ MB_written   vs base                │
Write_SingleEntry-4            16.86 ± ∞ ¹   14.90 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4             46.02 ± ∞ ¹   51.50 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4             58.84 ± ∞ ¹   58.01 ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4              53.39 ± ∞ ¹   55.49 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4              63.96 ± ∞ ¹   73.55 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_Concurrent-4             9.045 ± ∞ ¹   9.403 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4   9.121 ± ∞ ¹   8.156 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4      28.37 ± ∞ ¹   28.67 ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_ConcurrentReadWrite-4    2.944 ± ∞ ¹   3.018 ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                        21.72         22.00        +1.29%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │    old.bench    │                new.bench                │
                                                       │ entries_written │ entries_written  vs base                │
Write_SingleEntry-4                                         145.4k ± ∞ ¹      128.5k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_SmallBatch-4                                          522.9k ± ∞ ¹      585.3k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                          601.6k ± ∞ ¹      593.1k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_HugeBatch-4                                           510.0k ± ∞ ¹      530.0k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_MegaBatch-4                                           600.0k ± ∞ ¹      690.0k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                                 13.45k ± ∞ ¹      14.47k ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/NoCompression_Single-4          67.81k ± ∞ ¹      77.62k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4        9.495k ± ∞ ¹      8.556k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4         124.9k ± ∞ ¹      103.3k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4       9.060k ± ∞ ¹     10.100k ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4       108.70k ± ∞ ¹      87.20k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4      9.400k ± ∞ ¹      9.300k ± ∞ ¹       ~ (p=0.700 n=3) ²
Write_Concurrent-4                                          111.7k ± ∞ ¹      116.1k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               109.90k ± ∞ ¹      98.27k ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                   363.7k ± ∞ ¹      367.5k ± ∞ ¹       ~ (p=1.000 n=3) ²
Write_ConcurrentReadWrite-4                                 36.84k ± ∞ ¹      37.76k ± ∞ ¹       ~ (p=0.100 n=3) ²
geomean                                                     91.23k            90.71k        -0.58%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │   old.bench   │             new.bench              │
                                                       │     B/op      │     B/op       vs base             │
Write_SingleEntry-4                                       162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                         947.0 ± ∞ ¹     876.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                       8.727Ki ± ∞ ¹   8.655Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                        84.39Ki ± ∞ ¹   84.33Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                        835.3Ki ± ∞ ¹   835.2Ki ± ∞ ¹  ~ (p=0.400 n=3) ²
Write_CompressibleEntries-4                              6.572Ki ± ∞ ¹   6.415Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     1.257Ki ± ∞ ¹   1.206Ki ± ∞ ¹  ~ (p=0.300 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4        940.0 ± ∞ ¹     871.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    11.99Ki ± ∞ ¹   11.72Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4     8.914Ki ± ∞ ¹   8.853Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   119.0Ki ± ∞ ¹   119.2Ki ± ∞ ¹  ~ (p=0.700 n=3) ²
Write_Concurrent-4                                        162.00 ± ∞ ¹     90.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                               188.0 ± ∞ ¹     116.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                  515.0 ± ∞ ¹     443.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                                701.0 ± ∞ ¹     629.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                 2671.0 ± ∞ ¹     120.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                  3.701Ki ± ∞ ¹   2.099Ki ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                  72.00 ± ∞ ¹      0.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                   460.0 ± ∞ ¹     373.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                  2.143Ki                        ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │  old.bench   │            new.bench             │
                                                       │  allocs/op   │  allocs/op   vs base             │
Write_SingleEntry-4                                       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_SmallBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_LargeBatch-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_HugeBatch-4                                         6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_MegaBatch-4                                         9.000 ± ∞ ¹   7.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressibleEntries-4                               17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Single-4        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Single-4      8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch10-4       6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4     17.00 ± ∞ ¹   15.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/NoCompression_Batch100-4      6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4    108.0 ± ∞ ¹   106.0 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_Concurrent-4                                        6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentMultiShard-4                              7.000 ± ∞ ¹   5.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentBatched-4                                 6.000 ± ∞ ¹   4.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Write_ConcurrentReadWrite-4                               8.000 ± ∞ ¹   6.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_SequentialRead-4                                38.000 ± ∞ ¹   1.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_RandomAccess-4                                   45.00 ± ∞ ¹   18.00 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_LagMeasurement-4                                 2.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
Consumer_MultiConsumer-4                                  5.000 ± ∞ ¹   3.000 ± ∞ ¹  ~ (p=0.100 n=3) ²
geomean                                                   9.264                      ?               ³ ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ summaries must be >0 to compute geomean
⁴ ratios must be >0 to compute geomean

                                                       │   old.bench   │               new.bench               │
                                                       │ MB_compressed │ MB_compressed  vs base                │
Write_CompressibleEntries-4                                3.637 ± ∞ ¹     3.913 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4       4.937 ± ∞ ¹     4.441 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4      4.708 ± ∞ ¹     5.240 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4     4.894 ± ∞ ¹     4.841 ± ∞ ¹       ~ (p=0.400 n=3) ²
geomean                                                    4.510           4.582        +1.60%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │  old.bench  │              new.bench              │
                                                       │ MB_original │ MB_original  vs base                │
Write_CompressibleEntries-4                              5.172 ± ∞ ¹   5.564 ± ∞ ¹       ~ (p=0.200 n=3) ²
Write_CompressionComparison/WithCompression_Single-4     7.947 ± ∞ ¹   7.161 ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4    7.583 ± ∞ ¹   8.454 ± ∞ ¹       ~ (p=0.400 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4   7.891 ± ∞ ¹   7.807 ± ∞ ¹       ~ (p=0.700 n=3) ²
geomean                                                  7.042         7.161        +1.69%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                       │     old.bench     │                 new.bench                 │
                                                       │ compression_ratio │ compression_ratio  vs base                │
Write_CompressibleEntries-4                                   703.0m ± ∞ ¹        703.3m ± ∞ ¹       ~ (p=0.800 n=3) ²
Write_CompressionComparison/WithCompression_Single-4          618.9m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=0.900 n=3) ²
Write_CompressionComparison/WithCompression_Batch10-4         620.9m ± ∞ ¹        619.8m ± ∞ ¹       ~ (p=0.100 n=3) ²
Write_CompressionComparison/WithCompression_Batch100-4        620.3m ± ∞ ¹        620.1m ± ∞ ¹       ~ (p=0.900 n=3) ²
geomean                                                       639.8m              639.9m        +0.01%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │     old.bench      │               new.bench               │
                            │ entries_compressed │ entries_compressed  vs base           │
Write_CompressibleEntries-4         13.45k ± ∞ ¹         14.47k ± ∞ ¹  ~ (p=0.200 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                            │      old.bench       │                new.bench                │
                            │ entries_uncompressed │ entries_uncompressed  vs base           │
Write_CompressibleEntries-4            0.000 ± ∞ ¹            0.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │    old.bench     │              new.bench              │
                          │ MB_per_iteration │ MB_per_iteration  vs base           │
Consumer_SequentialRead-4       63.00m ± ∞ ¹       63.00m ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │       old.bench       │                new.bench                 │
                          │ entries_per_iteration │ entries_per_iteration  vs base           │
Consumer_SequentialRead-4            1.000k ± ∞ ¹            1.000k ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │    old.bench     │              new.bench              │
                        │ entries_per_read │ entries_per_read  vs base           │
Consumer_RandomAccess-4        5.000 ± ∞ ¹        5.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │      old.bench       │                new.bench                │
                          │ expected_lag_entries │ expected_lag_entries  vs base           │
Consumer_LagMeasurement-4            500.0 ± ∞ ¹            500.0 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                         │    old.bench     │              new.bench              │
                         │ active_consumers │ active_consumers  vs base           │
Consumer_MultiConsumer-4        3.000 ± ∞ ¹        3.000 ± ∞ ¹  ~ (p=1.000 n=3) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

Benchmarks run on Ubuntu latest with Go 1.24.5

jaredLunde added 8 commits August 6, 2025 15:25

checkpoint-ish

6af8c1d

checkpoint-ish

9cf7682

checkpoint-ish

bae0e3b

checkpoint-ish

b5fd2e1

checkpoint-ish

833f3d1

checkpoint-ish

b7a074f

checkpoint-ish

d9c0f8d

checkpoint-ish

2aa9d93

fix flush not updating index

e66de38

fix flush not updating index

4e68b52

fix flush not updating index

f67d1d0

fix flush not updating index

d8a794b

fix flush not updating index

113afcf

fix flush not updating index

8609909

fix flush not updating index

55d2596

fix flush not updating index

492c690

fix flush not updating index

338d218

fix flush not updating index

c896ee6

fix flush not updating index

86a25cf

fix flush not updating index

5dfac5e

fix flush not updating index

f9e75f1

fix flush not updating index

0045db6

fix flush not updating index

7737289

fix flush not updating index

34a59b5

fix flush not updating index

98e9567

checkpoint-ish

70ea29e

fix flush not updating index

974bb06

fix flush not updating index

68fe42f

fix flush not updating index

e57e0d5

fix flush not updating index

5338f66

fix flush not updating index

8a28560

fix flush not updating index

0187e27

jaredLunde merged commit 916a0b6 into main Aug 7, 2025
4 checks passed

jaredLunde deleted the next branch August 7, 2025 23:34

fix realtime #14

fix realtime #14

Uh oh!

Conversation

jaredLunde commented Aug 7, 2025

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results

Uh oh!

github-actions bot commented Aug 7, 2025

Comet Benchmark Results