Skip to content

Commit 4610bb4

Browse files
paskalumputun
paskal
authored and
umputun
committed
Add readme explanation for the benchmarks
1 parent 7e6b244 commit 4610bb4

File tree

2 files changed

+159
-5
lines changed

2 files changed

+159
-5
lines changed

‎README.md‎

Lines changed: 56 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -74,12 +74,59 @@ func main() {
7474
}
7575
```
7676

77-
### v3 performance improvements
77+
### Performance Comparison
7878

79-
v3 (and v2) are done using generics and 38-42% faster than v1 without them according to benchmarks.
79+
For detailed benchmarks comparing different versions and cache implementations, see the [benchmarks](./benchmarks) directory.
80+
81+
Based on all the benchmarks across four different caching libraries:
82+
83+
1. **[go-pkgz/expirable-cache](https://github.com/go-pkgz/expirable-cache)** remains the best overall option:
84+
- Excellent performance across all operations
85+
- Lowest memory usage and allocations
86+
- Type safety through generics
87+
- Clean API with method chaining
88+
- Simple implementation
89+
90+
2. **[dgraph-io/ristretto](https://github.com/dgraph-io/ristretto)** is a strong contender for specific use cases:
91+
- Great performance for read-heavy workloads
92+
- Sophisticated memory management for very large caches
93+
- Built-in metrics and statistics
94+
- Designed for high-concurrency environments
95+
96+
3. **[patrickmn/go-cache](https://github.com/patrickmn/go-cache)** is still fastest for pure raw performance but lacks modern features, and leaks goroutines
97+
98+
4. **[jellydator/ttlcache](https://github.com/jellydator/ttlcache)** lags behind in performance compared to all other options.
99+
100+
#### Version Improvements
101+
102+
v2 and v3 use Go generics and achieve significant performance improvements over v1:
103+
104+
- v2 is approximately **38-42% faster** than v1 for basic operations
105+
- v3 maintains the performance gains of v2 while being compatible with the Hashicorp `simplelru` interface
106+
107+
#### Performance Comparison
108+
109+
| Operation | v1 | v2 | v3 | Improvement v1→v3 |
110+
|-------------------------|-----------|-----------|-----------|-------------------|
111+
| Random LRU (no expire) | 272.4 ns/op | 160.1 ns/op | 158.1 ns/op | ~42% faster |
112+
| Frequency LRU (no expire) | 261.6 ns/op | 152.8 ns/op | 150.9 ns/op | ~42% faster |
113+
| Random LRU (with expire) | 286.5 ns/op | 177.6 ns/op | 175.3 ns/op | ~39% faster |
114+
| Frequency LRU (with expire) | 279.6 ns/op | 170.3 ns/op | 168.1 ns/op | ~40% faster |
115+
116+
#### Cross-Library Comparison
117+
118+
Recent benchmarks comparing expirable-cache with other popular Go caching libraries:
119+
120+
| Operation | [go-pkgz/expirable-cache](https://github.com/go-pkgz/expirable-cache) | [patrickmn/go-cache](https://github.com/patrickmn/go-cache) | [jellydator/ttlcache](https://github.com/jellydator/ttlcache) | [dgraph-io/ristretto](https://github.com/dgraph-io/ristretto) |
121+
|-----------|-----------------|----------|----------|-----------|
122+
| Set | 69.65 ns/op | 84.63 ns/op | 430.7 ns/op | 793.8 ns/op |
123+
| Get | 78.27 ns/op | 66.29 ns/op | 193.1 ns/op | 82.61 ns/op |
124+
| Set+Get | 67.69 ns/op | 68.97 ns/op | 242.8 ns/op | 197.6 ns/op |
125+
| Real-world scenario | 79.98 ns/op | 70.60 ns/op | 200.0 ns/op | 85.88 ns/op |
126+
| Memory allocations | Lowest | Low | Medium | Highest |
80127

81128
<details>
82-
<summary>v1</summary>
129+
<summary>v1 benchmark results</summary>
83130

84131
```
85132
~/expirable-cache ❯ go test -bench=.
@@ -120,7 +167,7 @@ ok github.com/go-pkgz/expirable-cache 18.307s
120167
</details>
121168

122169
<details>
123-
<summary>v3</summary>
170+
<summary>v3 benchmark results</summary>
124171

125172
```
126173
~/Desktop/expirable-cache/v3 master !2 ❯ go test -bench=.
@@ -158,4 +205,8 @@ BenchmarkLRU_Freq_WithExpire-8 7098261 168.1 ns/op
158205
PASS
159206
ok github.com/go-pkgz/expirable-cache/v3 24.315s
160207
```
161-
</details>
208+
</details>
209+
210+
<details>
211+
212+
For detailed benchmarks and methodology, see the [benchmarks directory](./benchmarks).

‎benchmarks/README.md‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# expirable-cache benchmarks
2+
3+
This directory contains comprehensive benchmarks comparing performance across different caching libraries for Go.
4+
5+
## Libraries Compared
6+
7+
1. **[go-pkgz/expirable-cache](https://github.com/go-pkgz/expirable-cache)** (v3) - This library, uses generics and LRU/LRC eviction
8+
2. **[patrickmn/go-cache](https://github.com/patrickmn/go-cache)** - Lightweight in-memory key:value store/cache with expiration support
9+
3. **[jellydator/ttlcache](https://github.com/jellydator/ttlcache)** - An in-memory cache with expiration
10+
4. **[dgraph-io/ristretto](https://github.com/dgraph-io/ristretto)** - A high performance memory-bound Go cache from Dgraph
11+
12+
## Benchmark Results
13+
14+
Here are the results from running the benchmarks on an Apple M3 processor:
15+
16+
```
17+
$ go test -bench=. -benchmem
18+
goos: darwin
19+
goarch: arm64
20+
pkg: github.com/go-pkgz/expirable-cache/benchmarks
21+
cpu: Apple M3
22+
BenchmarkGoCache_Set-8 13150846 84.63 ns/op 68 B/op 1 allocs/op
23+
BenchmarkGoCache_Get-8 17746971 66.29 ns/op 3 B/op 0 allocs/op
24+
BenchmarkGoCache_SetAndGet-8 17334808 68.97 ns/op 36 B/op 1 allocs/op
25+
BenchmarkTTLCache_Set-8 2818526 430.7 ns/op 4 B/op 0 allocs/op
26+
BenchmarkTTLCache_Get-8 6197169 193.1 ns/op 51 B/op 1 allocs/op
27+
BenchmarkTTLCache_SetAndGet-8 4923871 242.8 ns/op 28 B/op 1 allocs/op
28+
BenchmarkExpirableCache_Set-8 17001220 69.65 ns/op 4 B/op 0 allocs/op
29+
BenchmarkExpirableCache_Get-8 15427915 78.27 ns/op 3 B/op 0 allocs/op
30+
BenchmarkExpirableCache_SetAndGet-8 17600006 67.69 ns/op 4 B/op 0 allocs/op
31+
BenchmarkRistretto_Set-8 1502190 793.8 ns/op 262 B/op 5 allocs/op
32+
BenchmarkRistretto_Get-8 14158246 82.61 ns/op 27 B/op 2 allocs/op
33+
BenchmarkRistretto_SetAndGet-8 6852046 197.6 ns/op 96 B/op 2 allocs/op
34+
BenchmarkGoCache_GetWithTypeAssertion-8 17942817 67.64 ns/op 3 B/op 0 allocs/op
35+
BenchmarkTTLCache_GetWithoutTypeAssertion-8 6029751 199.0 ns/op 51 B/op 1 allocs/op
36+
BenchmarkExpirableCache_GetWithoutTypeAssertion-8 15133410 79.33 ns/op 3 B/op 0 allocs/op
37+
BenchmarkRistretto_GetWithTypeAssertion-8 14187883 86.12 ns/op 27 B/op 2 allocs/op
38+
BenchmarkGoCache_RealWorldScenario-8 16314248 70.60 ns/op 4 B/op 0 allocs/op
39+
BenchmarkTTLCache_RealWorldScenario-8 5891400 200.0 ns/op 52 B/op 1 allocs/op
40+
BenchmarkExpirableCache_RealWorldScenario-8 14579878 79.98 ns/op 3 B/op 0 allocs/op
41+
BenchmarkRistretto_RealWorldScenario-8 12897400 85.88 ns/op 28 B/op 2 allocs/op
42+
```
43+
44+
## Summary of Results
45+
46+
| Operation | [go-pkgz/expirable-cache](https://github.com/go-pkgz/expirable-cache) | [patrickmn/go-cache](https://github.com/patrickmn/go-cache) | [jellydator/ttlcache](https://github.com/jellydator/ttlcache) | [dgraph-io/ristretto](https://github.com/dgraph-io/ristretto) |
47+
|-----------|-----------------|----------|----------|-----------|
48+
| Set | 69.65 ns/op | 84.63 ns/op | 430.7 ns/op | 793.8 ns/op |
49+
| Get | 78.27 ns/op | 66.29 ns/op | 193.1 ns/op | 82.61 ns/op |
50+
| Set+Get | 67.69 ns/op | 68.97 ns/op | 242.8 ns/op | 197.6 ns/op |
51+
| Real-world scenario | 79.98 ns/op | 70.60 ns/op | 200.0 ns/op | 85.88 ns/op |
52+
| Memory allocations (Set) | 4 B/op | 68 B/op | 4 B/op | 262 B/op |
53+
| Memory allocations (Get) | 3 B/op | 3 B/op | 51 B/op | 27 B/op |
54+
55+
## Analysis
56+
57+
1. **[go-pkgz/expirable-cache](https://github.com/go-pkgz/expirable-cache)**:
58+
- Best overall balance of performance and features
59+
- Fastest Set operations among all libraries
60+
- Very competitive Get operations
61+
- Lowest memory usage across all benchmarks
62+
- Type safety through generics
63+
- Clean API with method chaining
64+
65+
2. **[patrickmn/go-cache](https://github.com/patrickmn/go-cache)**:
66+
- Fastest Get operations
67+
- Very competitive overall performance
68+
- However, it's known to leak goroutines and lacks modern features
69+
- Higher memory usage for Set operations than expirable-cache
70+
71+
3. **[dgraph-io/ristretto](https://github.com/dgraph-io/ristretto)**:
72+
- Excellent for read-heavy workloads
73+
- Much higher memory usage than other libraries
74+
- Considerably slower Set operations
75+
- Best suited for very large caches where sophisticated memory management is beneficial
76+
77+
4. **[jellydator/ttlcache](https://github.com/jellydator/ttlcache)**:
78+
- Significantly slower than other libraries for all operations
79+
- Higher memory usage for Get operations
80+
- Not recommended for performance-critical applications
81+
82+
Thanks to [@analytically](https://github.com/analytically) for the benchmark code and initial analysis!
83+
84+
## Running the Benchmarks
85+
86+
To run the benchmarks yourself:
87+
88+
```bash
89+
go test -bench=. -benchmem
90+
```
91+
92+
For more focused testing:
93+
94+
```bash
95+
# Test only Set operations
96+
go test -bench=Set -benchmem
97+
98+
# Test only expirable-cache
99+
go test -bench=ExpirableCache -benchmem
100+
101+
# Test only real-world scenarios
102+
go test -bench=RealWorldScenario -benchmem
103+
```

0 commit comments

Comments
 (0)