ollama timeout value

[sys1aeg]

getting client timeouts using call to local ollama. Is there a timeout value in config to increase
log from ollama:
level=INFO source=runner.go:682 msg="aborting completion request due to client closing the connection"

[inkeep-ai-assistant[bot]]

The Frigate configuration doesn't currently expose a timeout setting for Ollama requests(1)(2). The default timeout appears to be 60 seconds, which can cause issues during model pulls or inference on CPU-only systems(3).

Based on the GitHub discussion, there are a few recommendations:

1. Pre-download your model - Download the model to your Ollama instance before starting Frigate (e.g., run ollama pull llava:7b on your Ollama server)(3)

2. Use GPU acceleration - CPU-only Ollama is not recommended as inference times make it impractical for use with Frigate(1). On CPU, inference can take over 60 seconds and may not complete(3)

3. Configure Ollama for parallel requests - Set environment variables like OLLAMA_NUM_PARALLEL=1 and choose appropriate OLLAMA_MAX_QUEUE and OLLAMA_MAX_LOADED_MODELS values for your hardware(1)

The "aborting completion request due to client closing the connection" message suggests Frigate is timing out before Ollama completes the inference. Without GPU acceleration, this is expected behavior with the current implementation(3).

If you need to use CPU-only Ollama, you may want to consider using one of the cloud-based GenAI providers (Gemini, OpenAI, Azure OpenAI) instead, or stick with Frigate's built-in Jina CLIP model for semantic search(1)(3).

📚 Sources:

1. Generative AI
2. Full Reference Config
3. [Development]: 0.15 Ollama - timeout errors on init due to model pull taking > 60 seconds

---

Have another question? Just tag @inkeep.

[NickM-27]

You can control this in 0.17 using provider_options

[cmmh]

I'm sorry, I'm just finding finding it in the docs anywhere. What provider_option increases the timeout value that Frigate waits before closing the socket?

[NickM-27]

For Ollama I believe it is options.keep_alive but it's best to just check Ollama docs

[sys1aeg]

I am running llama on a large windows image. My problem may be I did not install the CUDA dev kit to take advantage of the GPUs on the graphics card.

[jason-e-gross]

nvtop and nvidia-smi (i think) are the two tools you can use to see if the GPU is being used during its usage.

[NickM-27]

The easiest way is to look at ollama. Use ollama ps and it will show how much of the model is on the GPU

[sys1aeg]

Thanks guys !

[Matzebhv]

I have switched to the new qwen3.5:35b and now the problem appears her too from time to time now. Modell is preloaded completly in vram. Problem is, if a person walks tru our yard more than one cameras detects this and the llm has a hard time to generate all the stuff for alerts and descriptions. It would be helpfull if frigates waits a little longer for the response of the llm.

[NickM-27]

The timeout is 2 minutes, if the response takes longer than 2 minutes then that is far too slow to be usable.

[Matzebhv]

Ok, 2 minutes should be long enough. Oh man, one problem here was, frigate is firing events into the llm and also HA llm-vision does. I have forgotten to change the model in llm-vision to qwen3.5 so everytime it fires up it unloads 3.5 and loads 3.0 into memory. Question, at this time no qwen3.5.instruct model is available. Am i able to set the nothink-option under provider options? think: false wont work.

[NickM-27]

You disable thinking with --chat-template-kwargs = {"enable_thinking": false}