Agents do not work with Self-hosted Models

[sreevatsank1999]

What happened?

Agents endpoint attempts to use the OpenAI backend when set to use a Self-Hosted model, and fails with MODEL_NOT_FOUND error as llama3.3:70b is not a valid OpenAI model.

It works as expected when using OpenAI Model or Gemini Model

Steps to Reproduce

1. Create an Agent with Ollama model
2. Ask a query

What browsers are you seeing the problem on?

Microsoft Edge

Relevant log output

{"attemptNumber":1,"headers":{"content-length":"18","content-type":"text/plain","date":"Mon, 27 Jan 2025 19:25:00 GMT"},"lc_error_code":"MODEL_NOT_FOUND","level":"error","message":"[handleAbortError] AI response error; aborting request: 404 404 page not found\n\nTroubleshooting URL: https://js.langchain.com/docs/troubleshooting/errors/MODEL_NOT_FOUND/\n","pregelTaskId":"07ee7040-9216-559c-adc5-e35920184572","retriesLeft":6,"stack":"Error: 404 404 page not found\n\nTroubleshooting URL: https://js.langchain.com/docs/troubleshooting/errors/MODEL_NOT_FOUND/\n\n    at APIError.generate (/app/node_modules/@langchain/openai/node_modules/openai/error.js:54:20)\n    at OpenAI.makeStatusError (/ap...","status":404,"timestamp":"2025-01-27T19:25:00.738Z"}

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

It works with Ollama models if you name your endpoint "Ollama".

Otherwise, it needs to be complete OpenAI compatible as a custom endpoint, did you set a baseURL?

Please share your librechat.yaml to troubleshoot.

[sreevatsank1999]

That is right. The endpoint is not names "Ollama", and I do not want the endpoint to display "Ollama" in the UI I want a different name. And yes the baseURL is set correctly as these models work in a non-agent setting.

[rnavarroz]

Hi @sreevatsank1999 , did you figure out how resolve it?, I am getting similar problem for self-hosted models, now the normal conversations with the AI goes directly ephemeral agents, it's very visible the issue.

[sreevatsank1999]

@rnavarroz No I stopped using this feature