Add custom endpoint URL support for Judge LLM in evaluations

[gorkem-bwl]

The Judge LLM in LLM Evaluations currently only supports cloud API providers (OpenAI, Anthropic, Google, xAI, Mistral, OpenRouter) etc .. Customers with internally hosted models (Ollama, vLLM, TGI, or any OpenAI-compatible inference server) cannot use them as the Judge LLM.

As of now model being evaluated supports endpointUrl field for custom/local endpoints however Judge LLM only accepts provider, model, apiKey eg no endpoint URL option. Users should be able to provide a custom endpoint URL for the Judge LLM, enabling self-hosted models to serve as the judge in evaluations.

Files that need changes:

FE:

• Clients/src/presentation/pages/EvalsDashboard/NewExperimentModal.tsx — Add optional endpointUrl field to the judgeLlm config object and render
  an input for it
• Related TypeScript interfaces for the judge config

BE:

• EvalServer/src/utils/run_evaluation.py (~line 145-172) — Extract endpointUrl from judge config and set OPENAI_API_BASE environment variable when present
• EvaluationModule/src/deepeval_engine/deepeval_evaluator.py — Update get_judge_llm() and CustomDeepEvalLLM to accept and pass base_url
• EvaluationModule/src/deepeval_engine/model_runner.py — Update _setup_openai() to use base_url parameter when OPENAI_API_BASE is set
• EvaluationModule/scorers/judge_runner.py — Pass base_url when creating OpenAI client for judge scoring
• EvaluationModule/scorers/provider_registry.py — Allow custom models when a custom endpoint is provided (relax the hardcoded model allowlist)

[erikankrom]

I'm using LiteLLM as an AI Gateway to inference hosted on multiple provider platforms including Microsoft AI Foundry, Amazon Bedrock, and Google Vertex AI. Other AI Gateways include platforms like Portkey. These are a single OpenAI-compatible endpoint with routing/fallback to different model providers to support auditing, quotas, chargeback/showback, guardrails, and centralized access to all "approved" models. Any location where an AI key may be required in Verifywise (AI Advisor, Evals, AI Detection, etc.) it would be useful to include the generic option for an OpenAI-compatible API endpoint. The model selection will be dynamic, so you'll need to load it at runtime vs pre-populated with options like (recommended) in the selection dropdown. You can provide a recommendation on the form entry instead.

[gorkem-bwl]

We have a custom endpoint for judge LLM but havent tried if that works with Litellm/portkey or similar LLM gateways. I dont think it'd be an issue but we may need to be more explicit here that they can enter any OpenAI compatible endpoitns here.

For AI Advisor, yes, that is something I missed - we have Anthropic, OpenAI, OpenRouter options (and even suggested LLMs for best p/p ratio of inference) but no OpenAI compatible endpoints to handle Portkey/Litellm or similar. I'll create an issue now and implement this in the next release.

Using the same key in AI-Detection should be there in the codebase, and will be in the next release, and it also uses the same LLM key you enter in the settings.

Let me know if you have other thoughts about implementation?

[gorkem-bwl]

OK, let me add the custom headers support issue now and we can work on it - quite straightforward. We can add a simple (optional) "Custom headers" key-value input under the Endpoint URL field when "Custom / Self-hosted" is selected. It wont be in this week's release but in 2 weeks' time (we do releases every 2 weeks).

[gorkem-bwl]

There is now #3404 - you can pull latest and check.

This is for custom endpoints for LLM keys (advisor, ai-detection, and AI agents in the very near future)