Skip to content

Commit 3133b1d

Browse files
Sela TachnaiSela Tachnai
authored andcommitted
Fix interface + OpenAI URL compatibility + automate all test sets in one go
1 parent 1be6481 commit 3133b1d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

68 files changed

+6256
-15270
lines changed

README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ A multi-platform, reproducible tool for benchmarking the **local** energy consum
1212

1313
- 🤖 **Local LLM Support**
1414
- **Ollama**: Connect to your local Ollama instance
15+
- **OpenAI-Compatible**: Connect to vLLM, AnythingLLM, or other OpenAI-compatible APIs
1516
- More providers coming soon (llama.cpp, etc.)
1617

1718
- 📊 **Comprehensive Metrics**
@@ -111,6 +112,29 @@ Options:
111112
--help Show this message and exit
112113

113114
The runner looks for `testset_<name>.json` inside `src/data/testsets/`. Passing `easy`, `testset_easy`, or the exact filename stem all point to the same file.
115+
116+
### OpenAI-Compatible Providers (vLLM, AnythingLLM, etc.)
117+
118+
To use an OpenAI-compatible provider instead of Ollama:
119+
120+
```bash
121+
python src/main.py run-test \
122+
--model "meta-llama/Meta-Llama-3-8B-Instruct" \
123+
--provider openai \
124+
--base-url "http://localhost:8000/v1" \
125+
--api-key "sk-optional"
126+
```
127+
128+
### Automation Script
129+
130+
To run all 4 standard test sets (easy, medium, hard, math) in sequence:
131+
132+
```bash
133+
python run_all_tests.py --model llama3:latest
134+
```
135+
136+
This will generate output files with the format `results/output_{model}_{dataset}_{date}.json`.
137+
114138
```
115139

116140
## Configuration

0 commit comments

Comments
 (0)