Skip to content

Commit 32aa02f

Browse files
authored
Update README.md
1 parent 9378c2f commit 32aa02f

File tree

1 file changed

+11
-0
lines changed

1 file changed

+11
-0
lines changed

README.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -468,6 +468,16 @@ Authorization: Bearer your_secret_api_key
468468

469469
## SOTA results on benchmarks with optillm
470470

471+
### AutoThink on GPQA-Diamond & MMLU-Pro (May 2025)
472+
473+
| **Model** | **GPQA-Diamond** | | **MMLU-Pro** | |
474+
|----------------|-----------------------------|--------------------------|----------------------------|--------------------------|
475+
| | Accuracy (%) | Avg. Tokens | Accuracy (%) | Avg. Tokens |
476+
| DeepSeek-R1-Distill-Qwen-1.5B | 21.72 | 7868.26 | 25.58 | 2842.75 |
477+
| with Fixed Budget | 28.47 | 3570.00 | 26.18 | 1815.67 |
478+
| **with AutoThink** | **31.06** | **3520.52** | **26.38** | **1792.50** |
479+
480+
471481
### LongCePO on LongBench v2 (Apr 2025)
472482

473483
| Model¹ | Context window | Short samples (up to 32K words) | Medium samples (32–128K words) |
@@ -552,6 +562,7 @@ called patchflows. We saw huge performance gains across all the supported patchf
552562
![Results showing optillm mixture of agents approach used with patchflows](https://raw.githubusercontent.com/codelion/optillm/main/moa-patchwork-results.png)
553563

554564
## References
565+
- [AutoThink: efficient inference for reasoning LLMs](https://dx.doi.org/10.2139/ssrn.5253327) - [Implementation](optillm/autothink)
555566
- [CePO: Empowering Llama with Reasoning using Test-Time Compute](https://cerebras.ai/blog/cepo) - [Implementation](optillm/cepo)
556567
- [LongCePO: Empowering LLMs to efficiently leverage infinite context](https://cerebras.ai/blog/longcepo) - [Implementation](optillm/plugins/longcepo)
557568
- [Chain of Code: Reasoning with a Language Model-Augmented Code Emulator](https://arxiv.org/abs/2312.04474) - [Inspired the implementation of coc plugin](optillm/plugins/coc_plugin.py)

0 commit comments

Comments
 (0)