Llama 3.1 Tool calling gets stuck in an infinite loop

[brentmid]

What happened?

Issue Description

Web search functionality with disableStreaming: true configuration hits the recursion limit (25) and fails to complete, returning a "Recursion limit reached" error instead of search results.

Environment

• LibreChat Version: v0.8.0-rc1
• Configuration: Custom endpoint with disableStreaming: true
• Tool: Web search (ephemeralAgent.web_search = true)

Steps to Reproduce

1. Configure a custom endpoint in librechat.yaml with disableStreaming: true
2. Enable web search in the LibreChat UI (globe icon)
3. Send a query that would trigger web search: "What's the latest news about AI?"
4. Observe the recursion error instead of search results

Expected Behavior

Web search should execute and return search results, regardless of streaming configuration.

Actual Behavior

An error occurred while processing the request: Recursion limit of 25 reached without hitting a stop condition. You can increase the limit by setting the "recursionLimit" config key.

Root Cause Analysis

This seems to be an architectural limitation in how LibreChat handles tool execution in non-streaming vs streaming modes:

Streaming Mode (Works)

• Tool calls are processed naturally during the streaming response flow
• Built-in flow control prevents excessive recursive calls
• Web search executes and integrates results seamlessly

Non-Streaming Mode (Fails)

• Tool calls are processed synchronously via handleToolCalls() in ModelEndHandler (line 44 in /api/server/controllers/agents/callbacks.js)
• Each web search result triggers a new agent step, incrementing the recursion counter
• No proper completion detection for web search in synchronous execution mode
• Hits recursion limit (25) before search process can complete properly

Technical Evidence

Key Code Path:
/api/server/controllers/agents/callbacks.js:43-45

if (metadata.provider === Providers.GOOGLE || graph.clientOptions?.disableStreaming) {
  handleToolCalls(data?.output?.tool_calls, metadata, graph);
}

The Problem:

1. disableStreaming: true triggers immediate synchronous tool execution
2. Web search results create additional agent processing steps
3. Each step increments the recursion counter
4. Process hits limit before natural completion

Impact

This affects users who:

• Use custom endpoints with disableStreaming: true (required for some LLM proxy setups)
• Need web search functionality
• Cannot use streaming mode due to infrastructure constraints

Potential Solutions

Option 1: Streaming Override for Web Search

Force streaming mode when web search is enabled, regardless of disableStreaming configuration.

Pros: Simple fix, preserves working web search
Cons: Overrides user configuration

Option 2: Improve Non-Streaming Tool Completion Detection

Implement proper completion signaling for web search in non-streaming mode to prevent recursive loops.

Pros: Respects all configurations, comprehensive fix
Cons: Complex architectural change

Related Work

This issue was discovered while fixing the core disableStreaming configuration bug (PR #9021). The core streaming configuration now works correctly, but this architectural limitation affects web
search specifically.

Additional Context

• Web search works in streaming mode
• This is specifically a non-streaming mode architectural constraint
• The recursion counter is configured in /api/server/controllers/agents/client.js:745
• Affects all web search providers (SearXNG, etc.) equally

Version Information

localhost/librechat debug a5b67510002a About an hour ago 1.29 GB
ghcr.io/danny-avila/librechat v0.8.0-rc1 ec025d2f48db 8 days ago 1.29 GB

% git rev-parse HEAD

d1c7a65

Steps to Reproduce

1. Configure a custom endpoint in librechat.yaml with disableStreaming: true
2. Enable web search in the LibreChat UI (globe icon)
3. Send a query that would trigger web search: "What's the latest news about AI?"
4. Observe the recursion error instead of search results

What browsers are you seeing the problem on?

Chrome

Relevant log output

Error Message from LibreChat Logs:

  2025-08-13 03:17:00 [31merror[39m: [31m[api/server/controllers/agents/client.js #sendCompletion] Operation aborted Recursion limit of 25 reached without hitting a stop condition. You can increase
  the limit by setting the "recursionLimit" config key.

  Troubleshooting URL: https://langchain-ai.github.io/langgraphjs/troubleshooting/errors/GRAPH_RECURSION_LIMIT/

  2025-08-13 03:17:00 [31merror[39m: [31m[api/server/controllers/agents/client.js #sendCompletion] Unhandled error type Recursion limit of 25 reached without hitting a stop condition. You can increase
   the limit by setting the "recursionLimit" config key.

  Troubleshooting URL: https://langchain-ai.github.io/langgraphjs/troubleshooting/errors/GRAPH_RECURSION_LIMIT/

  Configuration Evidence:

  LibreChat configuration correctly loaded with:
  "disableStreaming": true
  "webSearch": {
    "searchProvider": "searxng",
    "searxngInstanceUrl": "${SEARXNG_INSTANCE_URL}",
    "safeSearch": 0,
    "scraper": "firecrawl",
    "reranker": "jina"
  }

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

This is an issue with the Llama model and the underlying service specifically. it's well known that smaller local models have recursive problems, see:
https://discuss.huggingface.co/t/tool-calling-gets-stuck-in-an-infinite-loop/149792