locally hosted ollama model is not recognied with suggested input file

[krishnapitike]

I am using the following script to run paperqa. However it looks like paperqa is not recognizing that I am asking it to use ollama model. The error is given below the input file:

Models hosted with ollama are also supported. To run the example below make sure you have downloaded llama3.2 and mxbai-embed-large via ollama.

from paperqa import Settings, ask

local_llm_config = {
"model_list": [
{
"model_name": "ollama/llama3.2",
"litellm_params": {
"model": "ollama/llama3.2",
"api_base": "http://localhost:11434",
},
}
]
}

answer_response = ask(
"What is PaperQA2?",
settings=Settings(
llm="ollama/llama3.2",
llm_config=local_llm_config,
summary_llm="ollama/llama3.2",
summary_llm_config=local_llm_config,
embedding="ollama/mxbai-embed-large",
),
)

error:

       AuthenticationError: litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the  
       client or by setting the OPENAI_API_KEY environment variable. Received Model Group=gpt-4o-2024-11-20                                                             
       Available Model Group Fallbacks=None                                                                                                                             

[15:47:11] Generating answer for 'What is PaperQA2?'.
Status: Paper Count=0 | Relevant Papers=0 | Current Evidence=0 | Current Cost=$0.0000
Answer: I cannot answer this question due to insufficient information.

[dosubot]

Hi @krishnapitike! I'm Dosu and I’m helping the paper-qa team.

The error happens because the Ollama model config is missing "api_type": "ollama" in the litellm_params for both your LLM and embedding models. Without this, PaperQA (via litellm) defaults to OpenAI and expects an OpenAI API key, which triggers the authentication error you see. This applies to both the main LLM and the embedding model—if either is missing the correct config, PaperQA will try to use OpenAI for that part and fail if no key is set [source] [source].

Here's how to fix it:

local_llm_config = {
    "model_list": [
        {
            "model_name": "ollama/llama3.2",
            "litellm_params": {
                "model": "ollama/llama3.2",
                "api_base": "http://localhost:11434",
                "api_type": "ollama",
            },
        }
    ]
}

embedding_llm_config = {
    "model_list": [
        {
            "model_name": "ollama/mxbai-embed-large",
            "litellm_params": {
                "model": "ollama/mxbai-embed-large",
                "api_base": "http://localhost:11434",
                "api_type": "ollama",
            },
        }
    ]
}

answer_response = ask(
    "What is PaperQA2?",
    settings=Settings(
        llm="ollama/llama3.2",
        llm_config=local_llm_config,
        summary_llm="ollama/llama3.2",
        summary_llm_config=local_llm_config,
        embedding="ollama/mxbai-embed-large",
        embedding_config=embedding_llm_config,
    ),
)

Make sure:

• Ollama is running and both models are downloaded locally.
• The api_base URL matches your Ollama server.
• Every litellm_params dict (for LLM and embedding) includes "api_type": "ollama".

If you still see errors, double-check that all config blocks (including embedding) have the correct settings. More config examples and troubleshooting tips are in the docs and issues if you need them.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}