Agents: Tools and File Search not working with together.ai endpoint

[kantic]

What happened?

Hi,

First of all, I would like to thank you for this outstanding and easy-to-use piece of software!
I really like the "one-click" deployment options for Zeabur, Railway, etc., which makes it very easy to get a fully fledged LLM + RAG + Agent setup running (even for me, who is completely new to this)!

I came across a problem with agents when using the together.ai provider (e.g. using the model "meta-llama/Llama-3.3-70B-Instruct-Turbo").
When trying to use tools like the Google Search or Tavily Search tool, or when enabling file search, I get the following error in the UI:
"Something went wrong. Here's the specific error message we encountered: An error occurred while processing your request. Please contact the Admin."
Also, I get an error in the log file as given below.

When switching the model of the agent to an OpenAI model, e.g., "gpt4-turbo", everything works as expected, and no errors occur.

Steps to Reproduce

1. Setup LibreChat in Zeabur
2. Configure the together.ai endpoint
3. Create a new agent using, e.g., the model "meta-llama/Llama-3.3-70B-Instruct-Turbo"
4. Either: Upload a PDF file and check the box "Enable File Search" at the sidebar
5. And/Or: Add and configure Google Search or Tavily Search tool
6. Make a prompt using this agent

What browsers are you seeing the problem on?

Firefox

Relevant log output

(node:1) MaxListenersExceededWarning: Possible EventTarget memory leak detected. 11 abort listeners added to [AbortSignal]. MaxListeners is 10. Use events.setMaxListeners() to increase limit

(Use `node --trace-warnings ...` to show where the warning was created)

2025-02-02 17:16:13 error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 500 Internal server error

2025-02-02 17:16:13 error: [handleAbortError] AI response error; aborting request: 500 Internal server error

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Unhandled error type 500 Internal server error

It seems to be an error on Together.ai's end. I've had issues with their function calling as well, it would be best to reach out to their support to understand why it's failing, even though the model is listed here: https://docs.together.ai/docs/function-calling

[walbercardoso]

Same issue with openai API

[zeemy23]

+1 this came up with openai API

[danny-avila]

No issues with OpenAI + Tools + File Search