Jan.ai doesn't use provider, how do I connect to the api server with litellm?

[Jeffrey04]

So I am using jan.ai to manage my list of LLMs, and uses the built-in API server to host the LLM service. This is how I connect to it in Python with OpenAI library

client = OpenAI(
    base_url="http://localhost:1337/v1",
    api_key="",
)

completion = client.chat.completions.create(
    model="llama3-hermes-8b",
    messages=[
        {
            "role": "user",
            "content": "A prompt",
        }
    ],
)

I don't use litellm directly, but through dspy. However, when dspy tries to get completion through litellm, it does this under the hood

  File "/home/jeffrey04/Projects/guessme/.venv/lib/python3.13/site-packages/litellm/main.py", line 984, in completion
    model, custom_llm_provider, dynamic_api_key, api_base = get_llm_provider(
                                                            ~~~~~~~~~~~~~~~~^
        model=model,
        ^^^^^^^^^^^^
    ...<2 lines>...
        api_key=api_key,
        ^^^^^^^^^^^^^^^^
    )
    ^

and it insist I need to provide a LLM provider with the BadRequestError, is there a way to bypass the provider part?

[Jeffrey04]

I think i found the answer here

https://docs.litellm.ai/docs/providers/openai#using-openai-proxy-with-litellm