[BUG] Check model availability before running benchmark

[bhavishya-pohani]

I thought I had disabled the judges by not providing the judge name. Through some debugging I realized that the judge was pointed to llama3-70b. I have no API keys for this model.

The codebase should check the model availability + judge availability and clearly warn the user/stop the process if the model is not available instead of starting off with the run

[autumnCanTell]

I've set up llama3.3-70b as the judge model, but I couldn't reproduce the results from the paper. Did you succeed in reproducing them?

[bhavishya-pohani]

@autumnCanTell tried Claude sonnet 4.5 over SEARCH & saw much lower scores than reported(for 4 sonnet). Wondering if it's a split issue

[xansar]

+1. And i want to know what will happen when the judge api is failed