Agent stops working: "The AI service rejected the request due to an error. This could be caused by an invalid API key or an improperly formatted request."

[leondape]

What happened?

I've built multiple agents with tools (Dalle but also custom tools)
All Chats with various agents get this bug after a while. From that point on such a chat is completely broken.
You must start a new chat, otherwise you will always receive the error.

I cannot pin down what is causing it: Tool usage, long system prompts, or a specific model. For now I think it happens on all cases.
I have the feeling that the issue appears more on longer chats, but I am absolutely not sure about that.
I belive it hast nothing to do with any API key, as it just happens!

Steps to Reproduce

1. Create Agent with approx. 1000 Token Prompt instruction
2. Attach any Plugin
3. Chat for a while
4. wait for the error

What browsers are you seeing the problem on?

No response

Relevant log output

2024-11-29 13:44:50 error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 400 Invalid value for 'content': expected a string, got null.
2024-11-29 13:44:50 error: [handleAbortError] AI response error; aborting request: 400 Invalid value for 'content': expected a string, got null.

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Issues are for reproducible errors and I can’t reproduce this. However I will keep trying and handle the null content edge case.

try to reproduce it reliably. Are you switching agents mid conversation? Switching endpoints?

it could be a context window issue.

[leondape]

It's hard to reproduce!
I think the issue is my tools: The sometimes call the wrong method in the background with the wrong parameters. Sometimes the params are empty, because they are not needed for one action but for another they are.

Could such a thing be causing it?

[danny-avila]

Share the API spec and I can test if that's causing the issue, you may need to format it differently if that's the case

[MNLierman]

I dont' understand how I am the only other person with this issue. Brand new fresh Docker setup. Other containers work fine, other apps and services accept my OpenAI keys just fine. I know the difference between the tokens and the keys. I actually have a dev account and have API keys assigned to a project. The keys work and allow use of OpenAI chat just fine. Yours doesn't. Seems there may be a problem with your ability to save the save so that it can access it? it says "no_user_key" yet I very clearly and very carefully pasted the key in and saved it. When it didn't work, I went back and created a new API key, and it still didn't work.
Edit: ⭢ My solution is below.

2025-01-07 11:38:38 warn: Invalid conversation ID: null
2025-01-07 11:38:38 info: ---`saveMessage` context: undefined
2025-01-07 11:38:38 info: ---Invalid conversation ID Params: {
  "sender": "GPT-4o",
  "messageId": "[redacted]",
  "conversationId": null,
  "parentMessageId": "[redacted]",
  "unfinished": false,
  "error": true,
  "final": true,
  "text": "{\"type\":\"no_user_key\"}",
  "isCreatedByUser": false,
  "user": "[redacted]"
}

2025-01-07 11:45:45 warn: [OpenAIClient.chatCompletion][stream] API error
2025-01-07 11:45:45 error: [handleAbortError] AI response error; aborting request: 404 The model `gpt-4o` does not exist or you do not have access to it.
2025-01-07 11:46:27 warn: [OpenAIClient.chatCompletion][stream] API error
2025-01-07 11:46:27 error: [handleAbortError] AI response error; aborting request: 404 The model `gpt-4o` does not exist or you do not have access to it.
2025-01-07 11:46:37 warn: [OpenAIClient.chatCompletion][stream] API error
2025-01-07 11:46:37 error: [handleAbortError] AI response error; aborting request: 404 The model `gpt-4-turbo` does not exist or you do not have access to it.```

[danny-avila]

@MNLierman pasted which key and where? What environment variable did you use?

with docker, you usually have to stop the containers (docker compose down) and restart before .env file changes take effect.

[starascendin]

I also am running into this issue w/ agent using Calculator tool.

I have a valid openai api key, i was able to run chat completion w/ gpt-4o-mini.

I then made a Agent, using gpt-4o-mini, and added calculator tool, and when i query it i get this error

[danny-avila]

@starascendin There's not enough information to help you out. Using Official API, reverse proxy, etc? What exactly is the error in the logs? Would recommend opening a separate discussion topic for your specific issues:

does your agent config look like this?

[starascendin]

ah i found out, so i think i've had extra fields in the .env, perhaps like GOOGLE_API_KEY (i copied from my other env file) which may have caused this issue.

I since copied from .env.example and filled out the keys i needed, and things worked.

[MNLierman]

@MNLierman pasted which key and where? What environment variable did you use?

with docker, you usually have to stop the containers (docker compose down) and restart before .env file changes take effect.

Thank you, this was it. I'm a DevOps newcomer, so I appreciate your speedy reply. I should have replied sooner to let you know that this was the issue. For some reason, using the UI to add the OpenAI key didn't work.

1. I had to paste the key into the .env file and then use docker compose stop and docker compose start (down and up works too but will remove the containers, delete them, and repull the image). I would also like to add for someone who finds this through a search that the key is expected to be the WHOLE thing, including the prefix (sk-proj-) added at the beginning – either you choose the prefix or it's predefined for you and it must be the entire string.
2. Then, make sure you uncomment the models.
3. Finally, you cannot use the OpenAI API without at least $5 on your account to start with. They don't do the free thing anymore. There is no OpenAI free usage at all using the API keys. None. I created by OpenAI account a year ago and didn't use it, didn't know they stopped the free usage / free trial. Almost all the LLM APIs cost now, sadly, and they are making money off you while they do it.

Adding my API keys for the others works fine: Except for Bing, which requires the cookies field from dev tools, and that lasts about 24 hours – even though the cookies explicitly state an expiry of 14 days – Microsoft is using X-Headers to fingerprint the browser along with the User Agent. For example, the reason that using Edge gave you 30 turns in Copilot instead of 5 or 10 was not the User Agent (though they used that too) but the X-Headers. If LibreChat were to provide both of these to Bing, I bet that the Captcha would rarely be an issue.
@danny-avila – Do you have any thoughts on how I could pass these to Bing?

Copilot Studio and Azure AI API Access: Something else I learned, these are not similar in any way. I falsely believed that because of the name Copilot Studio, that I was going to get an API key after finishing the steps of deploying the bot app. Microsoft does not make this clear. I wasted the better part of 2 days doing this. Copilot Studio, included free with Office 365 Enterprise E3 and E5 licenses, allows biz owners to create a chat bot with auto-responder capabilities and some AI sprinkled in. They do not offer any API access to the bot you create, and it's so buggy that it's not worth attempting to use unless you have specific use cases.

[jescalan]

In case it helps at all, I was able to reproduce this error consistently with the following configuration. Using docker, no modifications to any of the files including .env. I have an openAI API key entered via the UI, which works for normal chat. Tried to create an agent that uses DALL-E, entered the same API key, and it gives me back this error each time.

[danny-avila]

uses DALL-E, entered the same API key, and it gives me back this error each time.

Also entered in the UI when adding tools?

[leondape]

I infer that you have entered an API Key for OpenAI, meaning for Chat Endpoints.
Does the error also occur if you set a specific DallE API Key in .env?

[leondape]

My initial issue was related to agents using tools. I have the feeling that the discussion has diverged from that since I am reading a lot about .env variables etc.
My issue was not related to API keys at all I believe.
My issue was due to the agent using a custom tool in a bad way with wrong parameters etc. This lead to the exception.
It was resolved by improving the tool descriptions.

Don't know it that helps or if the issues were related..

[mansiibm]

Hi, with the release- v0.7.6 I would like to bring to your notice after many steps of verification the following observation. I tried accessing my data I indexed with Azure AI Search via plugin and Agents :

1. The token usage is less (in 100s) using plugin. This method always works.
2. For same criteria, token usage for agents, is very high (crosses max context length of models 128K) and it Always fails after max follow up to 2 on chat.

Case 1: Plugins endpoint with azure ai search tool, I am able to get the results. And the backend logs look like-

Case 2: Agents with azure ai search tool -

it states some 127500 for max-context-tokens as soon as I query and the full token usage is as-

So how and why the tokens are being used up in agents like this?

And if on the same chat page I continue querying it goes into the api key error as shared below-

The relevant log status-

If I will refresh the page and again pass the same query it will fetch results.

Why is the token usage for same index at azure ai search and same query - looks very low for plugin and very high for agent which later fails with the error as shared. This issue is reproducible every time.

Can you please share the cause and solution for this.
This issue I have experienced with- OpenAI- gpt-4o and AzureOpenAI- gpt-4o models.

[ppatayane]

I am able to reproduce the issue too.

1. I call Azure AI Search from Plugin, it works. I further follow up in chat, all is fine. Never breaks
2. I call same tool Azure AI Search from Agents, after up to 2-3 follow up on chat I get error.
   

While using agents, also I tried setting max output tokens less than context window, resend files off(though I am attaching no files to chat), top P 10% and so on, nothing helps with any of the models using agent. It is same behavior using Open AI, Anthropic.. The error "The AI service rejected the request due to an error. This could be caused by an invalid API key or an improperly formatted request." comes not 1st time but after mostly 2 follow ups on chat.. which makes me think, it is not related to API key at all or request formation as well...Looking at logs, it exceeds token window

The similar logs as shared above for exceeding context window.. which surprisingly does not happen using Plugin.. It's the same use case I am trying on Agents using same provider Open AI(4o), anthropic(Sonnet3.5)

[maxesse]

This is also happening for me on my production instance with agents, but not with my dev box. It started happening after the commits on 30/01, on the very first message. Assistants and base models work fine.

{"attemptNumber":1,"code":null,"error":{"code":null,"message":"You didn't provide an API key. You need to provide your API key in an Authorization header using Bearer [REDACTED] (i.e. Authorization: Bearer YOUR_KEY), or as the password field (with blank username) if you're accessing the API from your browser and are prompted for a username and password. You can obtain an API key from https://platform.openai.com/account/api-keys.","param":null,"type":"invalid_request_error"},"headers":{"alt-svc":"h3=\":443\"; ma=86400","cf-cache-status":"DYNAMIC","cf-ray":"90a8f8f4ec882e0f-ARN","connection":"keep-alive","content-length":"496","content-type":"application/json; charset=utf-8","date":"Fri, 31 Jan 2025 10:21:02 GMT","server":"cloudflare","set-cookie":"__cf_bm=1IyFX9PktO.AheJueaFwPpqeqTCCyGOW0ECOwmBNWEs-1738318862-1.0.1.1-pgzG9zdXbwx4Piqr6D0FdwVPYDcOpSFtq2gD41.kHW_BTEAfu9Z0HOhw4M9U.5ga8oNpi6iQOYRmeKb3NmImPA; path=/; expires=Fri, 31-Jan-25 10:51:02 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None, _cfuvid=0G2gJtn9kf.jNco0Epff38FngBsqvRcHc26IRXA278A-1738318862039-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None","strict-transport-security":"max-age=31536000; includeSubDomains; preload","vary":"Origin","x-content-type-options":"nosniff","x-request-id":"req_e412a2fb6cefd744eed37a8f07e0e966"},"lc_error_code":"MODEL_AUTHENTICATION","level":"error","message":"[api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 401 You didn't provide an API key. You need to provide your API key in an Authorization header using Bearer auth (i.e. Authorization: Bearer YOUR_KEY), or as the password field (with blank username) if you're accessing the API from your browser and are prompted for a username and password. You can obtain an API key from https://platform.openai.com/account/api-keys.\n\nTroubleshooting URL: https://js.langchain.com/docs/troubleshooting/errors/MODEL_AUTHENTICATION/\n","param":null,"pregelTaskId":"1cc27248-81ab-5b02-82b9-69f56a63bbc8","request_id":"req_e412a2fb6cefd744eed37a8f07e0e966","retriesLeft":6,"stack":"Error: 401 You didn't provide an API key. You need to provide your API key in an Authorization header using Bearer auth (i.e. Authorization: Bearer YOUR_KEY), or as the password field (with blank username) if you're accessing the API from your browser and are prompted for a username and password. You can obtain an API key from https://platform.openai.com/account/api-keys.\n\nTroubleshooting URL: https://js.langchain.com/docs/troubleshooting/errors/MODEL_AUTHENTICATION/\n\n at APIError.generate (/app/node_modules/openai/error.js:48:20)\n at OpenAI.makeStatusError (/app/node_modules/openai/core.js:293:33)\n at OpenAI.makeRequest (/app/node_modules/openai/core.js:337:30)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async /app/node_modules/@librechat/agents/node_modules/@langchain/openai/dist/chat_models.cjs:1481:29\n at async RetryOperation._fn (/app/node_modules/p-retry/index.js:50:12)","status":401,"type":"invalid_request_error"}

Did you manage to work out what happened?

[danny-avila]

How is your API key configured?

[bvelasquez]

I was getting this error also after a clean installation today. My solution was to use gpt-3.5-turbo-0125 instead of using chatgpt-4o-latest. Therefore, in my case, it was selecting the incorrect or unsupported model when I setup my agent in the UI.

[MNLierman]

A lot of the agents written in the .env are not valid anymore. They get changed, deleted, and partially I believe that Google and OpenAI do this on purpose. I highly recommend using something other than GPT 3.5 due to cost. 4o significantly outperforms 3.5 Turbo in performance, knowledge, and pricing. Here is a list of the model names.

Taken from that models page: gpt-4o, gpt-4o-mini, gpt-4o-mini-2024-07-18, gpt-4o-2024-08-06, gpt-4o-2024-11-20, and o3-mini should all work with LibreChat.

Here is a performance doc comparison of 4o-mini vs 3.5 Turbo – 4o-mini beats 3.5 Turbo on every single test, including speed, accuracy, performance, and pricing.

OpenAI pages of interest:

• Valid model names 🡒
• Model names and technical specs/details, written by Microsoft 🡒
• Research updates & new model releases 🡒
• Model selector info 🡒

(Written for myself to refer back to as well.)

[keviddles]

I'm also seeing this on a fresh install, when trying to use the Calculator with an agent:

LibreChat         | 2025-02-12 22:32:05 error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type Received empty response from chat model call.
LibreChat         | 2025-02-12 22:32:05 error: [handleAbortError] AI response error; aborting request: Received empty response from chat model call.

[keviddles]

Using gpt-4o

[keviddles]

(My specific use case is I'm trying to set up mcp servers, but neither those nor the calculator tool are working; figured the calculator would be an easier use case to start with trying to get working)

[danny-avila]

@keviddles are you using OpenAI directly?

[keviddles]

Ooh great question - no I am not. I have a custom docker image that forwards from a proxy.

And that was absolutely the issue. I had to update my code to handle tool calls.

Working now - thank you @danny-avila !