[Bug]: [AskController] Error handling request fetch failed

[Leon-Sander]

What happened?

I have a model deployed via vllm, context size set to 20k.
When writing a larger amount of text in the librechat frontend, 5k tokens for example, I get the error:

error: [AskController] Error handling request fetch failed
error: [handleAbortError] AI response error; aborting request: fetch failed

I also dont see the request reaching my vllm container.
With less text there is no problem.

Previous discussions about this problem focused on proxy or reverse proxy, does not apply here I guess.

In the librechat.yaml:

endpoints:
  custom:
    - name: "vllm" # Internal identifier for this endpoint
      apiKey: "vllm" # Dummy API key (vLLM OpenAI endpoint often doesn't require one)
      baseURL: "http://host.docker.internal:8080/v1" # Corrected host, protocol, and standard base path
      models:
        # Replace "your-model-id-served-by-vllm" with the actual model ID
        # Example: "mistralai/Mistral-7B-Instruct-v0.1"
        default:
          - "/models/Qwen2.5-32B-Instruct-AWQ"
        # fetch: true will attempt to query the /v1/models endpoint
        fetch: false
      titleConvo: true
      # titleModel: "current_model" # Use the currently selected model (usually best)
      # To use a specific model for titles (must be in the default list or fetched):
      titleModel: "current_model"
      summarize: true
      summaryModel: "current_model" # Use the currently selected model if summarize were true
      forcePrompt: false
      modelDisplayLabel: "vLLM" # How the endpoint group appears in the UI

I tried setting fetch to false in hopes that it does not trigger the fetch, but still does it seems.

and in the .env for ollama as backend for an embedding model:

RAG_API_URL=http://host.docker.internal:8000
EMBEDDINGS_PROVIDER=ollama
OLLAMA_BASE_URL=http://host.docker.internal:11434
EMBEDDINGS_MODEL=nomic-embed-text

Version Information

docker images | grep librechat
ghcr.io/danny-avila/librechat-dev latest 07698c5407cb 2 weeks ago 950MB
ghcr.io/danny-avila/librechat-rag-api-dev latest 96061ddf7d66 6 weeks ago 7.81GB
ghcr.io/danny-avila/librechat-rag-api-dev-lite latest 817faebddae9 6 weeks ago 1.31GB

Steps to Reproduce

1. Setup vllm docker
2. Change librechat.yaml accordingly
3. Paste in a longer text.

What browsers are you seeing the problem on?

No response

Relevant log output

error: [AskController] Error handling request fetch failed
error: [handleAbortError] AI response error; aborting request: fetch failed

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[onestardao]

Looks like you’re hitting a fetch failure that’s not about the fetch at all — classic RAG-layer misdirection.

From the logs, your AskController dies on long inputs, but shorter ones pass — which probably means your context is fine, but your embedding fetch logic isn’t gating or streaming properly.

A few suspicious signals in what you posted:

    context_size = 20k, but no mention of how that’s enforced downstream

    You set fetch: false, but still get a fetch failed — so either (a) the system ignores the flag, or (b) something else still triggers a backend pull

    No chunking or streaming strategy mentioned — meaning your VM container probably gets one giant blob, chokes, and fails silent

You’re probably not hitting a network bug — you’re hitting a semantic misalignment between input volume and system memory flow.

If I were you, I’d look into:

    Making sure embedding fetches are streamable / gated at front

    Inspecting if tileModel or summaryModel logic still tries to make post-fetch calls even when fetch: false

    Verifying your middleware isn't expecting a pre-structured input that never arrives

It’s not just a Docker or endpoint thing — it’s probably a chunk handoff / prefetch ghost haunting you.