more tuning, benchmarking

Author: Thomas Stromberg

Makefile

@@ -1,4 +1,4 @@
-.PHONY: test lint bench clean
+.PHONY: test lint bench benchmark clean
 
 test:
 	go test -v -race -cover ./...
@@ -11,6 +11,14 @@ lint:
 bench:
 	go test -bench=. -benchmem
 
+benchmark:
+	@echo "Running benchmarks vs other Go cache libraries..."
+	@echo "Note: External libraries are only in benchmarks/ go.mod, not main go.mod"
+	@cd benchmarks && go test -bench=BenchmarkSpeed -benchmem -run=^$$
+	@echo ""
+	@echo "Running hit rate comparison (cherrypicked workload)..."
+	@cd benchmarks && go test -run=TestFIFOvsLRU_ScanResistance -v
+
 clean:
 	go clean -testcache
 

README.md

@@ -6,15 +6,9 @@
 [![Go Report Card](https://goreportcard.com/badge/github.com/codeGROOVE-dev/bdcache)](https://goreportcard.com/report/github.com/codeGROOVE-dev/bdcache)
 [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 
-Simple, fast, secure Go cache with [S3-FIFO eviction](https://s3fifo.com/) - better hit rates than LRU.
+<br clear="right">
 
-## Why?
-
-- **S3-FIFO Algorithm** - [Superior cache hit rates](https://s3fifo.com/) compared to LRU/LFU
-- **Fast** - ~20ns per operation, zero allocations
-- **Reliable** - Memory cache always works, even if persistence fails
-- **Smart Persistence** - Local files for dev, Cloud Datastore for Cloud Run
-- **Minimal Dependencies** - Only one (Cloud Datastore)
+Fast, persistent Go cache with S3-FIFO eviction - better hit rates than LRU, survives restarts with local files or Google Cloud Datastore, zero allocations.
 
 ## Install
 
@@ -35,28 +29,53 @@ if err := cache.Set(ctx, "answer", 42, 0); err != nil {
 }
 val, found, err := cache.Get(ctx, "answer")
 
-// With smart persistence (files for dev, Datastore for Cloud Run)
+// With smart persistence (local files for dev, Google Cloud Datastore for Cloud Run)
 cache, err := bdcache.New[string, User](ctx, bdcache.WithBestStore("myapp"))
 ```
 
 ## Features
 
 - **S3-FIFO eviction** - Better than LRU ([learn more](https://s3fifo.com/))
 - **Type safe** - Go generics
-- **Persistence** - Local files (gob) or Cloud Datastore (JSON)
+- **Persistence** - Local files (gob) or Google Cloud Datastore (JSON)
 - **Graceful degradation** - Cache works even if persistence fails
 - **Per-item TTL** - Optional expiration
 
 ## Performance
 
-Benchmarks from MacBook Pro M4 Max:
+### vs Popular Go Caches
+
+Benchmarks on MacBook Pro M4 Max comparing memory-only Get operations:
+
+| Library | Algorithm | ns/op | Allocations | Persistence |
+|---------|-----------|-------|-------------|-------------|
+| **bdcache** | S3-FIFO | **12.95** | **0 allocs** | ✅ Auto (Local files + GCP Datastore) |
+| golang-lru | LRU | 13.63 | 0 allocs | ❌ None |
+| otter | S3-FIFO | 15.73 | 0 allocs | ⚠️ Manual (Save/Load entire cache) |
+| ristretto | TinyLFU | 30.43 | 0 allocs | ❌ None |
+
+> ⚠️ **Benchmark Disclaimer**: These benchmarks are highly cherrypicked to show S3-FIFO's advantages. Different cache implementations excel at different workloads - LRU may outperform S3-FIFO in some scenarios, while TinyLFU shines in others. Performance varies based on access patterns, working set size, and hardware.
+>
+> **The real differentiator** is bdcache's automatic per-item persistence designed for unreliable environments like Cloud Run and Kubernetes, where shutdowns are unpredictable. See [benchmarks/](benchmarks/) for methodology.
+
+**Key advantage:**
+- **Automatic persistence for unreliable environments** - per-item writes to local files or Google Cloud Datastore survive unexpected shutdowns (Cloud Run, Kubernetes), container restarts, and crashes without manual save/load choreography
+
+**Also competitive on:**
+- Speed - comparable to or faster than alternatives on typical workloads
+- Hit rates - S3-FIFO protects hot data from scans in specific scenarios
+- Zero allocations - efficient for high-frequency operations
+
+### Detailed Benchmarks
 
-```
 Memory-only operations:
+```
 BenchmarkCache_Get_Hit-16      56M ops/sec    17.8 ns/op       0 B/op     0 allocs
 BenchmarkCache_Set-16          56M ops/sec    17.8 ns/op       0 B/op     0 allocs
+```
 
 With file persistence enabled:
+```
 BenchmarkCache_Get_PersistMemoryHit-16    85M ops/sec    11.8 ns/op       0 B/op     0 allocs
 BenchmarkCache_Get_PersistDiskRead-16     73K ops/sec    13.8 µs/op    7921 B/op   178 allocs
 BenchmarkCache_Set_WithPersistence-16      9K ops/sec   112.3 µs/op    2383 B/op    36 allocs

benchmarks/README.md

@@ -0,0 +1,105 @@
+# bdcache Benchmarks
+
+This directory contains comparison benchmarks against popular Go cache libraries.
+
+## ⚠️ Important Disclaimers
+
+### Cherrypicked Benchmarks
+
+These benchmarks are **intentionally cherrypicked** to demonstrate S3-FIFO's strengths:
+
+- **Scan resistance workloads** - Where large scans of cold data shouldn't evict hot working set
+- **One-hit wonder scenarios** - Where many items are accessed once and shouldn't pollute the cache
+- **Memory-only Get operations** - Pure speed comparisons without I/O
+
+**Different workloads favor different algorithms:**
+- **LRU** excels with temporal locality and simple sequential access patterns
+- **TinyLFU (Ristretto)** shines with frequency-based workloads and large caches
+- **S3-FIFO** handles mixed workloads with both hot items and one-hit wonders
+
+Your mileage **will** vary based on:
+- Access patterns (sequential, random, zipfian, etc.)
+- Working set size vs cache capacity
+- Read/write ratio
+- Key/value sizes
+- Hardware (CPU, memory speed)
+
+### The Real Differentiator: Persistence
+
+**bdcache's primary advantage isn't raw speed or hit rates** - it's the automatic per-item persistence designed for unreliable cloud environments:
+
+- **Cloud Run** - Instances shut down unpredictably after idle periods
+- **Kubernetes** - Pods can be evicted, rescheduled, or killed anytime
+- **Container environments** - Restarts lose all in-memory data
+- **Crash recovery** - Application failures don't lose cache state
+
+Other libraries require manual save/load of the entire cache, which:
+- Doesn't work when shutdowns are unexpected
+- Requires coordination and timing logic
+- Risks data loss on crashes
+- Adds operational complexity
+
+## Running Benchmarks
+
+### Speed Comparison
+
+```bash
+go test -bench=BenchmarkSpeed -benchmem
+```
+
+Compares raw Get operation performance across:
+- bdcache (S3-FIFO)
+- golang-lru (LRU)
+- otter (S3-FIFO with manual persistence)
+- ristretto (TinyLFU)
+
+### Hit Rate Comparison
+
+```bash
+go test -run=TestFIFOvsLRU_ScanResistance -v
+```
+
+Demonstrates S3-FIFO's scan resistance with a cherrypicked workload:
+1. Build 8K item working set (fits in 10K cache)
+2. Access working set once (marks as hot)
+3. One-time scan through 10K cold items
+4. Re-access working set (should hit)
+
+**Results:**
+- S3-FIFO: 100% hit rate (working set protected in Main queue)
+- LRU: 0% hit rate (scan evicted entire working set)
+
+This is a **best-case scenario for S3-FIFO**. Many real workloads won't see this dramatic of a difference.
+
+### Additional Tests
+
+```bash
+# S3-FIFO correctness tests
+go test -run=TestS3FIFO -v
+
+# Detailed behavior demonstrations
+go test -run=TestS3FIFODetailed -v
+
+# Hit rate comparisons (mixed workloads)
+go test -run=TestHitRateComparison -v
+```
+
+## Benchmark Files
+
+- `benchmark_comparison_test.go` - Speed benchmarks across libraries
+- `hitrate_comparison_test.go` - Hit rate workload generators
+- `fifo_vs_lru_test.go` - S3-FIFO vs LRU scan resistance demo
+- `s3fifo_debug_test.go` - Queue behavior validation
+- `s3fifo_detailed_test.go` - Detailed eviction order verification
+
+## Interpreting Results
+
+When evaluating caches for your use case:
+
+1. **Profile your actual workload** - Synthetic benchmarks don't capture real-world complexity
+2. **Measure what matters** - Hit rate, latency, throughput, memory usage
+3. **Consider operational needs** - Persistence, observability, graceful degradation
+4. **Test with your data** - Key/value sizes and access patterns vary wildly
+5. **Benchmark in production-like environments** - Hardware and load matter
+
+**Don't choose a cache based solely on these benchmarks.** Choose based on your specific requirements, with special attention to operational characteristics like persistence if you're running in unreliable cloud environments.