How to use it when there is only one model locally?

[Liuliusui]

My current issue is that my graphics memory is only sufficient to deploy one model locally, but the config requires two. How should I configure it to use one locally and one online? Because I see that only one api_base can be set in the config. Or how can I use two models with different api_bases?

[codelion]

You can use a single model by just putting the weight to 1.0. It is currently not possible to use different api_base but you can use an existing proxy or router like OptiLLM - https://github.com/codelion/optillm to router between models as needed with the same url.

[jvm123]

Different api_base should work for each model in the config.yml, just specify it for each model. The top level setting will only be used if not specified otherwise.

(cmp. the LLMModelConfig dataclass

openevolve/openevolve/config.py

Line 18 in 5d09222

api_base: str = None

and OpenAILLm that pulls it:

openevolve/openevolve/llm/openai.py

Line 40 in 5d09222

base_url=self.api_base,

; the shared setting for all models should not overwrite the individual settings as per the logic in

openevolve/openevolve/config.py

Line 99 in 5d09222

shared_config = {

)