feat(llm): Add LiteLLM (Proxy) Support

[par4m]

This PR aims to add support for LiteLLM as a Proxy to run multiple LLMs as mentioned in the issue #627

Defaults url to http://localhost:4000
To Test,

Install LiteLLM Proxy - pip install 'litellm[proxy]'

1. OpenAI, create config.yml

model_list:
  - model_name: "*"             # all requests where model not in your config go to this deployment
    litellm_params:
      model: openai/*           # set `openai/` to use the openai route
      api_key: os.environ/OPENAI_API_KEY

2. DeepSeek
   Run DeepSeek - ollama pull deepseek-r1
config.yml

model_list:
  - model_name: "deepseek-r1"
    litellm_params:
      model: "ollama_chat/deepseek-r1"
      api_base: "http://localhost:11434"

Run LiteLLM
litellm --config config.yml

In main.py set LLM Accordingly

cocoindex.functions.ExtractByLlm(
                llm_spec=cocoindex.LlmSpec(
                    api_type=cocoindex.LlmApiType.LITELLM,
                    model="deepseek-r1",  # or whatever model is set on your LiteLLM proxy
                    # address can be omitted to use http://0.0.0.0:4000 by default
                ),

[badmonster0]

Can you also help to update the documentation to describe how to use it there? Thanks!

[par4m]

Can you also help to update the documentation to describe how to use it there? Thanks!

Sure! Should I include both DeepSeek (ollama) and OpenAI examples ?

[badmonster0]

Can you also help to update the documentation to describe how to use it there? Thanks!

Sure! Should I include both DeepSeek (ollama) and OpenAI examples ?

Sounds good. Thanks!

[badmonster0]

Thanks for the PR!

[badmonster0]

Hi @par4m Param, Thank you so much for your contribution.
We made a section for you in our latest release note, we love your contribution and thank you ❤️!

https://cocoindex.io/blogs/cocoindex-changelog-2025-07-07/#par4m