Chat retrying multiple times

[amalthomasintel]

We have a custom endpoint configured and hosted llama on it.. While running the chat, it's always retrying 2 times. Any idea why it's calling the agent multiple times with the same function?

[danny-avila]

In general, this is a symptom of smaller LLM's when given tool definitions. Even the latest deepseek-chat models have this problem over API:

The current version of the deepseek-chat model's Function Calling capabilitity is unstable, which may result in looped calls or empty responses. We are actively working on a fix, and it is expected to be resolved in the next version.

source: https://api-docs.deepseek.com/guides/function_calling

qwen-2.5 models seem the best at tool calling for models you can easily run locally.