LiteLLM outputs gibberish

[frenzybiscuit]

What happened?

When using LiteLLM with Librechat it outputs gibberish.

I have tried adding dropParams presence_penalty as suggested elsewhere with no success.

When connected directly to the API backend, it works fine.

Version Information

e3b1c22a5a34

Steps to Reproduce

Add LiteLLM as a backend.

What browsers are you seeing the problem on?

Firefox

Relevant log output

.

Screenshots

.

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

LC doesn't submit default parameters like temperature, frequency penalty, presence_penalty, etc., only if you configure them via the UI or YAML.

You can get a closer look at what's being submitted in the debug logs, here are your log locations:

• Docker: Project root directory ./logs
• npm: ./api/logs

Debug logs are formatted like this: debug-YYYY-MM-DD.log

An example of the "gibberish" would also be helpful. I'm assuming you're attempting to use an image model directly via LC as configured via LiteLLM as proxy, but using image models directly like this isn't supported.

[frenzybiscuit]

So you're saying you have to set the parameters via the UI (or an agent) ?

I did this, and I'm getting gibberish. Here is an image of an agents response. It continues without stopping, as well. And answers itself.

The gibberish:

I will check the logs in a few moments.

[frenzybiscuit]

Nothing really stands out in the error logs.

[frenzybiscuit]

[frenzybiscuit]

When connected directly to the LLM backend (bypassing LiteLLM) it works fine. Example: