Disabling streaming does not work when tools are used

[matiashegoburu]

What happened?

I have setup a model that is served locally by llamacpp, which is notable for having the limitation of not being able to use tools when steaming is enabled. Adding a config to libre office to dropParams stream, and or addPArams stream: false, works when no tools are used, but the moment i enable tools, i get the error from llamacpp indicating that streaming is enabled.

Version Information

ghcr.io/danny-avila/librechat-dev latest 2ede259a0a67 2 days ago 1.03GB
ghcr.io/danny-avila/librechat-rag-api-dev-lite latest e7da8e6aca0f 9 days ago 1.31GB

Steps to Reproduce

serve a model with llamacpp
setup an mcp server tool
setup librechat custom endpoint with dropParams stream, addParams stream: false

What browsers are you seeing the problem on?

No response

Relevant log output

srv  log_server_r: request: POST /v1/chat/completions 172.17.0.1 500
got exception: {"code":500,"message":"Cannot use tools with stream","type":"server_error"}

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

For agents (which is what we use for tool usage), setting stream: false to addParams or dropping stream in dropParams will not work because the toggle for streaming is controlled by a separate mechanism.

There is an open issue to toggle off streaming:

#7297