[Question]: Serverless hosted endpoints failing with new version

[mansiibm]

Hi, I am experiencing an issue with the serverless supported models like - xai and deepseek models.

I am using the latest 7.9-rc1 image for my setup. Sharing a video snippet for better understanding.

serverless.endpoint.mp4

I have not got any tool activated or anything, just using the models from deepseek and xai and it stops working.

Also, one question- can I disable the file search from the normal chat via endpoints.

[mansiibm]

Logs in the backend.

[mansiibm]

I am using endpoints from azure.

[danny-avila]

endpoints:
  azureOpenAI:
    groups:
    - group: "Azure AI Deepseek"
      assistants: false 
      apiKey: "${AZURE_DEEPSEEK_API_KEY}"
      baseURL: "https://resource.region.models.ai.azure.com/v1/"
      version: "2024-08-01-preview"
      serverless: true
      models:
        DeepSeek-R1:
          deploymentName: "DeepSeek-R1"
    - group: "Azure AI Foundry"
      apiKey: "${LIBRECHAT_AZURE_OAI_API_KEY}"
      baseURL: "https://resource-region.services.ai.azure.com/models"
      version: "2024-05-01-preview"
      serverless: true
      dropParams: ["stream_options", "user"]
      models:
        Mistral-Large-2411:
          deploymentName: "Mistral-Large-2411"
        grok-3:
          deploymentName: "grok-3"

[mansiibm]

Hi Danny thanks for replying. I was setting up the models from xai and deepseek as endpoints(custom) so that they appear as providers to the user-

endpoints:
custom:
- name: "Deepseek"
apiKey: "${deepseek_api_key}"
baseURL: "https://region.api.cognitive.microsoft.com/models/chat/completions?api-version=2024-05-01-preview"
models:
default: ["DeepSeek-R1"]
fetch: false
titleConvo: true
directEndpoint: true
titleModel: "DeepSeek-V3-0324"
modelDisplayLabel: "Deepseek"
- name: "xai"
apiKey: "${xai_api_key}"
baseURL: "https://region.api.cognitive.microsoft.com/models/chat/completions?api-version=2024-05-01-preview"
models:
default: ["grok-3","grok-3-mini"]
fetch: false
titleConvo: true
directEndpoint: true
titleModel: "grok-3"
modelDisplayLabel: "xAI"

After this setup I am facing the issue when using the models to chat. I think it is happening because of the tools section in the chat menu.

Although no option is selected from the tool but still the model fails.

[mansiibm]

I even set up the way you suggested -

  - group: "serverless-example"
    apiKey: "api-key"
    baseURL: "https://region.api.cognitive.microsoft.com/models/chat/completions?api-version=2024-05-01-preview"
    version: "2024-05-01-preview"
    serverless: true
    models:
      DeepSeek-R1: true
      grok-3: true
      grok-3-mini: true

Now when I try creating an agent from azure openAI then the specific error I face is -

for deepseek -

I am not able to use the agent for deepseek with tools enabled.

I have to disable all the tools then it can respond to me

So this will give a bad user experience and I will not be able to remove the model also from the list, it will appear for the model list from azure for agents.

[gmerritt]

Same for me with litellm custom endpoints using the latest v0.7.9-rc1, built like an hour ago; here is my fresh message that was going to be a new issue, but I scanned the recent list and found this issue thread.

My description of this same situation:

---

Howdy all,

In builds at some point after post-v0.7.8 release, my LibreChat gives errors for any custom endpoint model in the general chat models menu if the endpoint is not also made explicitly available to agents via an entry in endpoints: agents: allowedProviders:. Here is a sample error when such a non-agent-endpoint model is selected in the general chat model menu:

The "Thinking" provider is not available for use with Agents. Please go to your agent's settings and select a currently available provider.

In this case, "Thinking" is a custom endpoint, via our LiteLLM, that bundles & presents "thinking" models.

Now, we totally have endpoints / models that we do not want to present as choices in Agents...because they do not work well with agents!

Is it right that now, basically, any & all custom endpoints must always be added to (and displayed in) the agents model picker, or they are 100% unsuable in all librechat contexts? If so, why is there even endpoints: agents: allowedProviders: anymore if you always have to include everything?

Or am I missing something? :D

Here is an example of one of our custom endpoints:

endpoints:
  custom:
    - name: "Thinking"
      apiKey: <redacted>
      baseURL: "https://<foo>"
      iconURL: "https://<bar>"
      models:
        default: ["deepseek.r1","grok-3-mini"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "Thinking"

Thanks!

[ppatayane]

The models we hosted are via Azure foundry but visible as a endpoint to user and not inside azure open ai.. This setup is not working with new release v0.7.9-rc1 @danny-avila

[mansiibm]

Hi any response/update/activity on this.....? just following up :)

[gmerritt]

This is still very, very much a problem as of the most-recent build (a9f01bb · yesterday).

If you do a generic, general chat in the main chat window and pick from the menu a model that is not associated with agents: allowedProviders: then game over, full stop, No Chat For You; 100% this error every time:

The "Azure OpenAI" provider is not available for use with Agents. Please go to your agent's settings and select a currently available provider.

So, if you want to offer any models for general chat but do not want them to appear as models available for agents, then...too bad, error time. ¯\(ツ)/¯

Here's my dev deployment's librechat.yaml purged of goodies, but should cover the bases:

# For more information, see the Configuration Guide:
# https://www.librechat.ai/docs/configuration/librechat_yaml

# Configuration version (required)
version: 1.2.1

cache: true

memory:
  disabled: true  # Set to true to completely disable memory
  personalize: true  # Gives users the ability to toggle memory on/off, true by default
  tokenLimit: 2000  # Maximum tokens for memory storage
  messageWindowSize: 5  # Number of recent messages to consider
  agent:
    provider: "✅ Agent models"
    model: "[OpenAI] GPT 4.1 Nano"

endpoints:
  custom:
    - name: "✅ Agent models"
      baseURL: "https://litellm.----------"
      models:
        default: ["gpt-4.1-nano","o4-mini","o3","gpt-3.5-turbo","gemini-2.5-flash","Gemini 2.5 Flash Lite","gemini-2.5-pro"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "✅ Agent models"
    - name: "Thinking"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/thinking.png"
      models:
        default: ["deepseek.r1","grok-3-mini"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "Thinking"
    - name: "Vision (receive images)"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/glasses.png"
      models:
        default: ["amazon.nova-pro-v1:0","gpt-4.1-nano","Gemini 2.5 Flash Lite","[Meta] Llama Maverick 17B Instruct","gemini-2.5-flash","gemini-2.5-pro","gpt-3.5-turbo","o3","o4-mini","claude-3-7-sonnet","claude-sonnet-4"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "Vision"
    - name: "Amazon"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/amazon.png"
      models:
        default: ["amazon.nova-pro-v1:0","amazon.nova-micro-v1:0"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "Amazon"
    - name: "Anthropic"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/anthropic.svg"
      models:
        default: ["claude-3-5-haiku","claude-3-7-sonnet","claude-sonnet-4"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "Anthropic"
    - name: "DeepSeek"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/deepseek.png"
      models:
        default: ["deepseek","deepseek.r1"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "DeepSeek"
    - name: "Google"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/gemini.png"
      models:
        default: ["gemini-2.5-flash","gemini-2.5-pro"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "Google"
    - name: "Inception"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/jais.png"
      models:
        default: ["[Inception] جبل جس (Jais 30B Chat)"]
        fetch: false
      maxContextTokens: 8000
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "Inception"
    - name: "Meta"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/meta.png"
      models:
        default: ["[Meta] Llama Maverick 17B Instruct","llama3-2-1b"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "Meta"
    - name: "Microsoft"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/microsoft.png"
      models:
        default: ["[Microsoft] Phi-4 Mini","[Microsoft] Phi-4 Multimodal"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "Microsoft"
    - name: "Mistral"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/mistral.png"
      models:
        default: ["mistral-7b","mistral-large-2407"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "Mistral"
    - name: "OpenAI"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/openai.png"
      models:
        default: ["gpt-4.1-nano","o3","o4-mini","gpt-3.5-turbo"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "OpenAI"
    - name: "xAI"
      baseURL: "https://litellm.----------"
      iconURL: "https://static.dev.----------/images/agent_icons/xai.svg"
      models:
        default: ["grok-3","grok-3-mini"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: true
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "xAI"
    - name: "LiteLLM-All"
      baseURL: "https://litellm.----------"
      models:
        default: ["[OpenAI] GPT 3.5 Turbo"]
        fetch: true
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: false
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "LiteLLM-All"
  agents:
    allowedProviders:
      - ✅ Agent models
      - Amazon
      - Anthropic
      - DeepSeek
      - Gemini 2.5 Flash Lite
      - Google
      - Inception
      - LiteLLM-All
      - Meta
      - Microsoft
      - Mistral
      - OpenAI
#      - Thinking
      - Vision (receive images)
    capabilities:
      - "file_search"
      - "actions"
      - "tools"
      - "chain"

fileConfig:
  serverFileSizeLimit: 16 # in MB
  avatarSizeLimit: 5 # in MB

#fileStrategy: "s3"

includedTools: ["calculator","dalle","google"]

interface:
  presets: false
  bookmarks: true
  runCode: false
  webSearch: false
  customWelcome: "BETA RELEASE ⇧ Select a model above to chat. Image generation, web search and a calculator are available by making a new agent with tools from the panel on the right. ⇨ BETA RELEASE"
  termsOfService:
    modalAcceptance: true
    modalTitle: "Terms of Service"
    modalContent: |
      Terms of Service
        BETA RELEASE: .......

rateLimits:
  fileUploads:
    ipMax: 100
    ipWindowInMinutes: 10
    userMax: 40
    userWindowInMinutes: 10
  conversationsImport:
    ipMax: 100
    ipWindowInMinutes: 60
    userMax: 20
    userWindowInMinutes: 60
  tts:
    ipMax: 100
    ipWindowInMinutes: 60
    userMax: 60
    userWindowInMinutes: 60
  stt:
    ipMax: 100
    ipWindowInMinutes: 60
    userMax: 60
    userWindowInMinutes: 60

secureImageLinks: false # Does not work with S3 CDN, only with the built-in file storage.

speech:
  stt:
    openai:
      model: 'whisper-1'
  tts:
    openai:
      model: 'tts-1'
      voices: ['alloy', 'ash', 'coral', 'echo', 'fable', 'nova', 'onyx', 'sage', 'shimmer']


modelSpecs:
  enforce: false
  prioritize: false
  addedEndpoints:
    - agents
    - Eco
    - Thinking
    - Vision (receive images)
    - Amazon
    - Anthropic
    - DeepSeek
    - Google
    - Inception
    - Meta
    - Microsoft
    - Mistral
    - OpenAI
    - xAI
    - LiteLLM-All
    - azureOpenAI
    - google
    - bedrock
  list:
    - name: "GPT 4.1 Nano"
      label: "GPT 4.1 Nano"
      default: true
      iconURL: "https://static.dev.----------/images/agent_icons/openai.png"
      description: "Good choice for general use. • Vision; 1M; May 2024; $0.10/M in, $0.40/M out"
      preset:
        endpoint: "OpenAI"
        model: "gpt-4.1-nano"
        modelLabel: "GPT 4.1 Nano"
        greeting: |
          GPT 4.1 Nano • Good choice for general use. • Vision; 1M; May 2024; $0.10/M in, $0.40/M out
    - name: "Gemini 2.5 Flash Lite"
      label: "Gemini 2.5 Flash Lite"
      default: false
      iconURL: "https://static.dev.----------/images/agent_icons/gemini.png"
      description: "Good choice for general use. • Vision; 1M; Jan 2025; $0.10/M in, $0.40/M out"
      preset:
        maxContextTokens: 1000000
        endpoint: "Google"
        model: "Gemini 2.5 Flash Lite"
        modelLabel: "Gemini 2.5 Flash Lite"
        greeting: |
          Gemini 2.5 Flash Lite • Good choice for general use. • Vision; 1M; Jan 2025; $0.10/M in, $0.40/M out

[gmerritt]

That's not custom; it's not mine, it's yours: https://www.librechat.ai/docs/configuration/dotenv#endpoints

But same error with (for example) my "Thinking" custom endpoint.

What drew me into this present issue is the error message, which may be a red herring. Would you like me to open a new issue and document it all from the start?

Thanks!

[danny-avila]

Ah I think I see what the issue is on your end now. You are seeing the "provider is not available for use with Agents" error despite not using "agents." Indeed allowedProviders should only be for use of agents explicitly and not when using the provider direct:

Indeed, this is a separate issue from OP, feel free to open a new one to make this more clear and for tracking.

[gmerritt]

Ok, will do but -- but -- but!!! -- I only put in direct provider to debug this issue with this exact error message when using custom endpoints that were not made available for agents. I wasn't using that at all, just put it in to test. Note I explicitly indicated above that my "Thinking" custom endpoint also makes this error.

Let me get my samples & error documentation all nice and clean for a new issue.

Thanks, Danny!

[gmerritt]

The robots tell me... (and, no, I do not trust them!):

Looking at lines 24-29 in createPayload.ts:

const endpoint = _e as s.EModelEndpoint;
let server = `${EndpointURLs[s.EModelEndpoint.agents]}/${endpoint}`;
if (s.isAssistantsEndpoint(endpoint)) {
  server =
    EndpointURLs[(endpointType ?? endpoint) as 'assistants' | 'azureAssistants'] +
    (isEdited ? '/modify' : '');
}

The bug is on line 24: The code always defaults to the agents endpoint URL regardless of which endpoint is actually selected. It only checks for assistants endpoints as a special case, but completely ignores custom endpoints and regular endpoints like openAI, anthropic, google, etc.

The Flow

• Frontend: When a user submits a message with a non-agent-eligible model (like gpt-3.5-turbo from openAI endpoint)
• createPayload(): Line 24 sets server = /api/agents/chat/openAI (incorrectly routing to agents)
• Backend: Request goes to /api/agents/chat/:endpoint route
• Validation: buildEndpointOption middleware calls agent initialization
• Error: agent.js:45-48 throws "invalid_agent_provider" because openAI is not in the agents allowedProviders

For non-agent endpoints, requests should go to:

• OpenAI models: /api/edit/openAI
• Anthropic models: /api/edit/anthropic
• Google models: /api/edit/google
• Custom models: /api/edit/custom

But the createPayload function is incorrectly routing everything through /api/agents/chat/{endpoint}.

The Fix

The createPayload.ts logic needs to be updated to properly route based on endpoint type...

[danny-avila]

It's wrong but on the right track, I'm already resolving this here: #8487

[danny-avila]

Closed by #8487

[mansiibm]

Hi Danny , thankyou for the fix, the tools and agents- seem to work with the serverless deployments

• group: "server-less models"
  apiKey: "${deepseek_api_key}"
  baseURL: "https://region.api.cognitive.microsoft.com/models/chat/completions?api-version=2024-05-01-preview"
  version: "2024-05-01-preview"
  serverless: true
  models:
  grok-3: true
  grok-3-mini: true
  DeepSeek-R1: true

But I had these models- grok-3 , grok-3-mini and deepseek deployed as custom endpoints initially as -

custom:
- name: "Deepseek"
apiKey: "${deepseek_api_key}"
baseURL: "https://region.api.cognitive.microsoft.com/models/chat/completions?api-version=2024-05-01-preview"
models:
default: ["DeepSeek-R1"]
fetch: false
titleConvo: true
directEndpoint: true
titleModel: "DeepSeek-V3-0324"
modelDisplayLabel: "Deepseek"
- name: "xai"
apiKey: "${xai_api_key}"
baseURL: "https://region.api.cognitive.microsoft.com/models/chat/completions?api-version=2024-05-01-preview"
models:
default: ["grok-3","grok-3-mini"]
fetch: false
titleConvo: true
directEndpoint: true
titleModel: "grok-3"
modelDisplayLabel: "xAI"

Now these custom endpoints are failing with the following error -

the same grok-3 model with the same base url and api key is working fine under azureOpenAI but not working under custom endpoint from now......

is it intended?