Commit de4fe89
refactor: remove EncodeParallel after benchmarks show 14% slower
Parallel encode was 14% slower on Bitcoin, 5-12x slower on small inputs,
and allocated 7x more memory. Per-match dictionary lookups are too fast
for parallelization overhead. Remove from public API and benchmarks.
Update benchmark results (93 benchmarks, clean run without parallel).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>1 parent beaef8b commit de4fe89
File tree
5 files changed
+254
-365
lines changed- benchmarks
- src
- benchmarks/Tiktoken.Benchmarks
- libs/Tiktoken.Core
5 files changed
+254
-365
lines changed
0 commit comments