Endpoint Error when message.content not a string

[Kristovich]

What happened?

Minimal Reproduction Using OpenRouter:

1. Create an agent with the Google tool (I used google/gemini-2.0-flash-001 as the model)
2. Ask it to search something
3. Switch to another LLM like meta-llama/llama-3.3-70b-instruct
4. Send any message

This will fail with
error: [handleAbortError] AI response error; aborting request: Provider returned error

A message history with a tool call also fails when using other openrouter llms, I guess it depends on whether the underlying provider supports it.

I was also able to get it to fail natively using the Perplexity Provider in LibreChat. Google, OpenAi, and Anthropic work properly.

I've narrowed it down; some providers doesn't like message.content to be anything other than a string.

Try it yourself using curl, or postman or something:
POST to https://openrouter.ai/api/v1/chat/completions

{
  "model": "meta-llama/llama-3.3-70b-instruct",
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "google",
        "description": "A search engine optimized for comprehensive, accurate, and trusted results. Useful for when you need to answer questions about current events.",
        "parameters": {
          "type": "object",
          "properties": {
            "query": {
              "type": "string",
              "minLength": 1,
              "description": "The search query string."
            },
            "max_results": {
              "type": "number",
              "minimum": 1,
              "maximum": 10,
              "description": "The maximum number of search results to return. Defaults to 10."
            }
          },
          "required": [
            "query",
            "max_results"
          ],
          "additionalProperties": false,
          "$schema": "http://json-schema.org/draft-07/schema#"
        }
      }
    }
  ],
  "messages": [
  {
    "role": "system",
    "content": "Use Google tool if needed"
  },
  {
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": "search something random for me real quick, I'm testing something"
      }
    ]
  },
  {
    "role": "assistant",
    "content": null,
    "tool_calls": [
      {
        "id": "tool_0_google",
        "type": "function",
        "function": {
          "name": "google",
          "arguments": "{\"query\":\"the history of rubber ducks\",\"max_results\":5}"
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": "[OUTPUT_OMITTED_FOR_BREVITY]",
    "name": "google",
    "tool_call_id": "tool_0_google"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "text",
        "text": "Okay, I searched for \"the history of rubber ducks\" and found some interesting results. It seems rubber ducks were invented in the late 19th century, and the first ones were solid and not intended to float.\n"
      }
    ]
  },
  {
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": "now search something else"
      }
    ]
  }
]
}

Yields

{"error":{"message":"Provider returned error","code":422,"metadata":{"raw":"{\"detail\":[{\"loc\":[\"body\",\"messages\",4,\"role\"],\"msg\":\"unexpected value; permitted: <ChatMessageRole.TOOL: 'tool'>\",\"type\":\"value_error.const\",\"ctx\":{\"given\":\"assistant\",\"permitted\":[\"tool\"]}},{\"loc\":[\"body\",\"messages\",4,\"content\"],\"msg\":\"str type expected\",\"type\":\"type_error.str\"},{\"loc\":[\"body\",\"messages\",4,\"tool_call_id\"],\"msg\":\"field required\",\"type\":\"value_error.missing\"},{\"loc\":[\"body\",\"messages\",4,\"content\"],\"msg\":\"str type expected\",\"type\":\"type_error.str\"},{\"loc\":[\"body\",\"messages\",4,\"role\"],\"msg\":\"unexpected value; permitted: <ChatMessageRole.USER: 'user'>\",\"type\":\"value_error.const\",\"ctx\":{\"given\":\"assistant\",\"permitted\":[\"user\"]}},{\"loc\":[\"body\",\"messages\",4,\"role\"],\"msg\":\"unexpected value; permitted: <ChatMessageRole.SYSTEM: 'system'>\",\"type\":\"value_error.const\",\"ctx\":{\"given\":\"assistant\",\"permitted\":[\"system\"]}},{\"loc\":[\"body\",\"messages\",4,\"content\"],\"msg\":\"str type expected\",\"type\":\"type_error.str\"}]}","provider_name":"DeepInfra","isDownstreamPipeClean":true,"isErrorUpstreamFault":true}},"user_id":"*********"}

Whereas

{
  "model": "meta-llama/llama-3.3-70b-instruct",
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "google",
        "description": "A search engine optimized for comprehensive, accurate, and trusted results. Useful for when you need to answer questions about current events.",
        "parameters": {
          "type": "object",
          "properties": {
            "query": {
              "type": "string",
              "minLength": 1,
              "description": "The search query string."
            },
            "max_results": {
              "type": "number",
              "minimum": 1,
              "maximum": 10,
              "description": "The maximum number of search results to return. Defaults to 10."
            }
          },
          "required": [
            "query",
            "max_results"
          ],
          "additionalProperties": false,
          "$schema": "http://json-schema.org/draft-07/schema#"
        }
      }
    }
  ],
  "messages": [
  {
    "role": "system",
    "content": "Use Google tool if needed"
  },
  {
    "role": "user",
    "content": "search something random for me real quick, I'm testing something"
  },
  {
    "role": "assistant",
    "content": null,
    "tool_calls": [
      {
        "id": "tool_0_google",
        "type": "function",
        "function": {
          "name": "google",
          "arguments": "{\"query\":\"the history of rubber ducks\",\"max_results\":5}"
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": "[OUTPUT_OMITTED_FOR_BREVITY]",
    "name": "google",
    "tool_call_id": "tool_0_google"
  },
  {
    "role": "assistant",
    "content": "Okay, I searched for \"the history of rubber ducks\" and found some interesting results. It seems rubber ducks were invented in the late 19th century, and the first ones were solid and not intended to float.\n"
  },
  {
    "role": "user",
    "content": "now search something else"
  }
]
}

Yields

{"id":"************","provider":"DeepInfra","model":"meta-llama/llama-3.3-70b-instruct","object":"chat.completion","created":1740263903,"choices":[{"logprobs":null,"finish_reason":"stop","native_finish_reason":"stop","index":0,"message":{"role":"assistant","content":"","refusal":null,"tool_calls":[{"index":0,"id":"***********","function":{"arguments":"{\"query\": \"what is the largest planet in our solar system\", \"max_results\": 5.0}","name":"google"},"type":"function"}]}}],"usage":{"prompt_tokens":443,"completion_tokens":27,"total_tokens":470}}

I was able to mostly fix it (Still fails on perplexity):
node_modules/@librechat/agents/node_modules/@langchain/openai/dist/chat_models.cjs

async completionWithRetry(request, options) {
(...)

                const { messages, ...others } = request
                const newMessages = messages
                    .map((message) => {
                        let textContent = message.content;
                        if (Array.isArray(message.content) && message.content.length === 1 && message.content[0].type === 'text') {
                            textContent = message.content[0].text;
                        }
                        return { ...message, content: textContent }
                    })
                const res = await this.client.chat.completions.create({ ...request, messages: newMessages }, requestOptions);

Version Information

# git rev-parse HEAD
12605516902179808d2dda495764a386438dcd62

Steps to Reproduce

1. Create an agent with the Google tool (I used google/gemini-2.0-flash-001 as the model)
2. Ask it to search something
3. Switch to another LLM like meta-llama/llama-3.3-70b-instruct
4. Send any message

What browsers are you seeing the problem on?

Firefox

Relevant log output

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Hi, thanks for the report.

You're going to have issues when you switch from a model that supports tool-calling, to one that doesn't, using the same agent configuration (in this case, adding the Google tool). We can't always know all the models that support tools or don't support them, so this is not something we can mitigate programmatically.

[Kristovich]

Thanks for the reply.

I just swapped back to the official docker image and started a brand new conversation. OpenRouter's meta-llama/llama-3.3-70b-instruct seems to be able to properly call tools in the first message, yet consistently fails on the second message.

I switched back to my hack fix and it begins to work again

The only difference between these 2 requests is that the first (failed) request is sending this as a message:

{
  role: string;
  content: {
    type: string;
    text: string;
  }[];
}

The second (successful) request is sending this as a message:

{
  role: string;
  content: string;
}

Which leads me to believe that some providers do not support arrays in the content block. Yet plain strings seems to be universally compatible.

Again, this is with the exact same model, with the same OpenRouter provider, with the same underlying service provider, DeepInfra. This doesn't seem to be a compatibility issue between different models, nor different providers, since the same model and provider (and subprovider) was used in this example.

It seems to be a formatting issue of the message array before its sent.

[xz-dev]

I agree, also in TheB.AI

[xz-dev]

Refer
#3115
https://community.openai.com/t/messages-1-content-is-invalid/700063/8

[xz-dev]

You can write a patch endpoint server to merge content as string, it's easy with AI

[danny-avila]

LibreChat already accounts for some providers that only accept content fields as strings.

What would be better is to add a configuration to tell LC to format the payload accordingly per-custom-endpoint as configurable.