Add LongBench V2 benchmark#249

Merged

RobotSail merged 12 commits intoinstructlab:mainfrom

eshwarprasadS:add-longbench

Jun 17, 2025

Commits on Jun 2, 2025

init add longbench benchmark implementation

eshwarprasadS
authored and
RobotSail
committed
add requirements file, fix optional dependency options, lint

eshwarprasadS
authored and
RobotSail
committed
feat: add requirements-cuda, move vllm and flash-attn to requirements-cuda extras

eshwarprasadS
authored and
RobotSail
committed
fix: fix openai-chat-completions model for enabling external / locally served openai-compatible model endpoints

eshwarprasadS
authored and
RobotSail
committed
fix: change model backend to local completions, add support for model name parameter

eshwarprasadS
authored and
RobotSail
committed
fix: linting...

eshwarprasadS
authored and
RobotSail
committed
linting changes...

eshwarprasadS
authored and
RobotSail
committed
fix: type changes and linting

eshwarprasadS
authored and
RobotSail
committed
add evaluator registration to evaluator group in unit test

eshwarprasadS
authored and
RobotSail
committed
typing fixes to accommodate API call args, linting...

eshwarprasadS
authored and
RobotSail
committed
fix: typing for py3.10

eshwarprasadS
authored and
RobotSail
committed

Commits on Jun 6, 2025

Merge branch 'main' into add-longbench
eshwarprasadS
authored