You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
| Router |`router`| Uses the [optillm-bert-uncased](https://huggingface.co/codelion/optillm-bert-uncased) model to route requests to different approaches based on the user prompt |
221
+
| Chain-of-Code |`coc`| Implements a chain of code approach that combines CoT with code execution and LLM based code simulation |
221
222
| Memory |`memory`| Implements a short term memory layer, enables you to use unbounded context length with any LLM |
222
223
| Privacy |`privacy`| Anonymize PII data in request and deanonymize it back to original value in response |
223
224
| Read URLs |`readurls`| Reads all URLs found in the request, fetches the content at the URL and adds it to the context |
### coc-claude-3-5-sonnet-20241022 on AIME 2024 pass@1 (Nov 2024)
295
+
296
+
| Model | Score |
297
+
|-------|-----:|
298
+
| o1-mini | 56.67 |
299
+
| coc-claude-3-5-sonnet-20241022 | 46.67 |
300
+
| coc-gemini/gemini-exp-1121 | 46.67 |
301
+
| o1-preview | 40.00 |
302
+
| f1-preview | 40.00 |
303
+
| gemini-exp-1114 | 36.67 |
304
+
| claude-3-5-sonnet-20241022 | 20.00 |
305
+
| gemini-1.5-pro-002 | 20.00 |
306
+
| gemini-1.5-flash-002 | 16.67 |
307
+
293
308
### readurls&memory-gpt-4o-mini on Google FRAMES Benchmark (Oct 2024)
294
309
| Model | Accuracy |
295
310
| ----- | -------- |
@@ -324,6 +339,7 @@ called patchflows. We saw huge performance gains across all the supported patchf
324
339
325
340
## References
326
341
342
+
-[Chain of Code: Reasoning with a Language Model-Augmented Code Emulator](https://arxiv.org/abs/2312.04474) - [Implementation](https://github.com/codelion/optillm/blob/main/optillm/plugins/coc_plugin.py)
327
343
-[Entropy Based Sampling and Parallel CoT Decoding](https://github.com/xjdr-alt/entropix) - [Implementation](https://github.com/codelion/optillm/blob/main/optillm/entropy_decoding.py)
328
344
-[Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation](https://arxiv.org/abs/2409.12941) - [Evaluation script](https://github.com/codelion/optillm/blob/main/scripts/eval_frames_benchmark.py)
329
345
-[Writing in the Margins: Better Inference Pattern for Long Context Retrieval](https://www.arxiv.org/abs/2408.14906) - [Inspired the implementation of the memory plugin](https://github.com/codelion/optillm/blob/main/optillm/plugins/memory_plugin.py)
0 commit comments