Add multi-model randomization#23

Open

dittops wants to merge 2 commits intomainfrom

codex/update-script-to-shuffle-between-models

Member

dittops commented May 27, 2025

Summary

allow listing multiple models in engine config YAML
randomize model choice per request in llmperf benchmark
support list of models through benchmark tooling
document multi-model option in README

Testing

python -m py_compile $(git ls-files '*.py')
pytest -q (fails: ModuleNotFoundError: No module named 'requests')


          Support multi-model runs via config

b4c32df

dittops added the codex label

— with

ChatGPT Codex Connector


          fix:

e211eab

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels