First message is dropped w/ MCP servers selected and custom endpoints

[KevWho]

What happened?

Some chat messages are not included in the API request whenever MCP servers are selected for a model hosted on a custom endpoint. This bug does not appear to occur for the default endpoints.

Version Information

ghcr.io/danny-avila/librechat-dev                             latest            b2acb22fc0ee   7 days ago      1.16GB
ghcr.io/danny-avila/librechat-rag-api-dev-lite                latest            833a22bcf14c   3 weeks ago     1.45GB

Steps to Reproduce

1. Add a custom endpoint to librechat.yaml and a MCP server. e.g.:

endpoints:
  openAI:
    titleModel: "gpt-4"
    plugins: true
    assistants: true
  custom:
    - name: "Custom"
      apiKey: "${CUSTOM_API_KEY}"
      # You can use the OpenAI endpoint here for easy repro, but any endpoint will do
      baseURL: "https://api.openai.com/v1"
      models:
        default:
          - "gpt-4"
        fetch: false
      modelDisplayLabel: "Custom"
  agents:
    recursionLimit: 200
    maxRecursionLimit: 500

mcpServers:
  grafana:
    type: "streamable-http"
    url: "${GRAFANA_MCP_URL}"
    initTimeout: 150000
    headers:
      Authorization: "Bearer ${GRAFANA_API_KEY}"

2. Select the MCP server from the chat dropdown for the custom endpoint's model and submit a query

What browsers are you seeing the problem on?

No response

Relevant log output

2025-08-05T23:37:56.008Z debug: [BaseClient] Loading history:
{
  conversationId: "e700c17b-fad9-4295-93b1-5e54ba516054",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
}
2025-08-05T23:37:56.195Z debug: [BaseClient] Context Count (1/2)
{
  remainingContextTokens: 7359,
  maxContextTokens: 7368,
}
2025-08-05T23:37:56.196Z debug: [BaseClient] Context Count (2/2)
{
  remainingContextTokens: 7359,
  maxContextTokens: 7368,
}
2025-08-05T23:37:56.196Z debug: [BaseClient] tokenCountMap:
{
  6a5ebfe8-ed66-49d6-8427-1b9683be2eff: 6,
}
2025-08-05T23:37:56.196Z debug: [BaseClient]
{
  promptTokens: 9,
  remainingContextTokens: 7359,
  payloadSize: 1,
  maxContextTokens: 7368,
}
2025-08-05T23:37:56.197Z debug: [BaseClient] tokenCountMap
{
  6a5ebfe8-ed66-49d6-8427-1b9683be2eff: 6,
}
2025-08-05T23:37:56.197Z debug: [BaseClient] userMessage
{
  messageId: "6a5ebfe8-ed66-49d6-8427-1b9683be2eff",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
  conversationId: "e700c17b-fad9-4295-93b1-5e54ba516054",
  sender: "User",
  text: "Foobar",
  isCreatedByUser: true,
  tokenCount: 6,
}
2025-08-05T23:37:56.435Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-08-05T23:37:56.646Z error: [api/server/controllers/agents/client.js #sendCompletion] Operation aborted 400 Invalid 'messages': empty array. Expected an array with minimum length... [truncated]
2025-08-05T23:37:56.647Z error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 400 Invalid 'messages': empty array. Expected an array with minimum len... [truncated]

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

You are running into a model context issue. GPT-4 has a max context window of about 8k and you can easily surpass this with MCP servers configured, which add a lot to the context. Use a different model and/or agent where you can configure the max context window as well as selectively attach individual tools from an MCP server to help manage context.

[KevWho]

Looks like this is indeed the issue. Adding presets for the custom models under modelSpecs to override the 4k token default resolves this.

[jcowles]

I'm seeing this too, particularly when an MCP provides a lot of context.

But it's odd that the failure mode is VLLM reporting that the messages list is empty, that seems like a bug.

(and it's not actually overflowing our model context window, it's some context limit internal to LibreChat that's being crossed, which then causes the chat mesages to be cleared -- the exact same input, with the same LLM, via OpenWebUI works fine)

[danny-avila]

VLLM reporting that the messages list is empty

LibreChat is pruning messages, as much as can fit in the determined context window. With a lower limit, it can be that no messages fit at all, hence the empty list. LC prefers to prune the list for you rather than send an "overflowing" list which can potentially lead to provider-side errors and/or costs.

it's some context limit internal to LibreChat that's being crossed

Correct. We can't know every limit to every LLM as they can vary by environment/provider, especially true of self-hosted models.

We have an open issue where we allow you to customize the limit by endpoint/model that we plan to implement:
#1633

You can also submit a PR for known limits for a model:

LibreChat/api/utils/tokens.js

Lines 238 to 242 in c465d7b

	// misc.
	kimi: 131000,
	// GPT-OSS
	'gpt-oss-20b': 131000,
	'gpt-oss-120b': 131000,

OWUI likely has a "known" model limit they are using or fetching directly from your provider somehow, as we do with OpenRouter which provides their context windows via API request.

To this end, what model are you using? What would be a reasonable limit to set for it? Is your context window configurable on vLLM side?

[JoeLudwig]

@jcowles and I are using Qwen3-235b-a22b-instruct-2507 with a context length limit of 131072. Other models are being added to and removed from the list all the time as new models come out and people want to test various workloads against them.

That number is indeed configurable in vLLM. vLLM provides max_model_len in response to its own .../models call, but that gets dropped once it is proxied through LiteLLM. (We're using LiteLLM to proxy models from multiple hosts because the list of models is so fluid.)

Would pointing LibreChat directly at vLLM without the proxy help? Would it use max_model_len instead of its own internal default?

[JoeLudwig]

vLLM returns this from models:

{
  "object": "list",
  "data": [
    {
      "id": "qwen3-235b-a22b-fp8",
      "object": "model",
      "created": 1755894639,
      "owned_by": "vllm",
      "root": "/data/git/Qwen3-235B-A22B-Instruct-2507-FP8",
      "parent": null,
      "max_model_len": 131072,
      "permission": [
        {
          "id": "modelperm-142cb66899a84a3988f9f62602850acf",
          "object": "model_permission",
          "created": 1755894639,
          "allow_create_engine": false,
          "allow_sampling": true,
          "allow_logprobs": true,
          "allow_search_indices": false,
          "allow_view": true,
          "allow_fine_tuning": false,
          "organization": "*",
          "group": null,
          "is_blocking": false
        }
      ]
    }
  ]
}

[jcowles]

definitely seems like a bug