You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
LiveCodeBench variants can crash during execution due to:
Dataset not being converted to list format
Missing type validation for examples
Reproduction Steps
# This can fail depending on dataset implementationfromeval.chat_benchmarks.LiveCodeBench.eval_instructimportLiveCodeBenchbenchmark=LiveCodeBench()
# Iteration may fail or behave unexpectedly