Custom Endpoint 400 (no body) error

[rwmehta]

I've just cloned the repo, so the only changes I've made have been to try and add a custom endpoint. I am running a model in Azure AI Foundry that is not OpenAI or AI Services, so I think a custom endpoint is the only way to connect. I am able to connect to it directly through from azure.ai.inference import ChatCompletionsClient, and from everything I see it is OpenAI API compatible. This is the error I see when trying to use it:
Something went wrong. Here's the specific error message we encountered: An error occurred while processing your request. Please contact the Admin.
And here is what is showing up in powershell:
LibreChat | 2025-02-10 17:08:49 warn: [OpenAIClient.chatCompletion][stream] API error LibreChat | 2025-02-10 17:08:49 error: LibreChat | 2025-02-10 17:08:49 error: [handleAbortError] AI response error; aborting request: 400 status code (no body).

Here is my setup in librechat.yaml:

cache: true
endpoints:
  custom:
    - name: "AZUREFOUNDRY"
      apiKey: "${AZUREFOUNDRY_KEY}"
      baseURL: "https://{instance_name}.eastus.models.ai.azure.com"
      models:
        default: ["Llama-3-3-70B-Instruct"]
        fetch: false
      titleConvo: true
      titleModel: "current_model"
      summarize: True
      summaryModel: "current_model"
      forcePrompt: false
      modelDisplayLabel: "Llama"

If I make forcePrompt true then I get this error:
LibreChat | 2025-02-10 17:36:24 error: [handleAbortError] AI response error; aborting request: Failed to send message. HTTP 404 - NOT FOUND

I also don't quite understand what the models field is. Is it appended onto the URL somewhere when making an API call?

[danny-avila]

Hard to say without your config, but you can also try via azure serverless config:

https://www.librechat.ai/docs/configuration/azure#serverless-inference-endpoints

[rwmehta]

What config would that be?

I tried the serverless method:

version: 1.1.4
cache: true
endpoints:
  azureOpenAI:
    groups:
    - group: "serverless-example"
      apiKey: "${AZUREFOUNDRY_KEY}"  # arbitrary env var name
      baseURL: "https://{instance_name}.eastus.models.ai.azure.com"
      # version: "2024-05-01-preview" # Optional: specify API version
      serverless: true
      models:
        # Must match the deployment name of the model
        Llama-3-3-70B-Instruct: true

And saw the same error in the UI:
Something went wrong. Here's the specific error message we encountered: An error occurred while processing your request. Please contact the Admin.

These are the errors in the console:

LibreChat | 2025-02-10 18:10:51 error: KeyvMongo connection error: connect ECONNREFUSED 172.18.0.2:27017 LibreChat | 2025-02-10 18:10:51 error: There was an uncaught error: connect ECONNREFUSED 172.18.0.2:27017

LibreChat | 2025-02-10 18:13:13 warn: [OpenAIClient.chatCompletion][stream] API error LibreChat | 2025-02-10 18:13:13 error: LibreChat | 2025-02-10 18:13:13 error: [handleAbortError] AI response error; aborting request: 400 status code (no body)

I do have a serverless deployment in Azure AI Foundry, but the endpoint I have is different from the format I can see in their docs:
They expect:
https://<resource-name>.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview

While I have:
https://<resource-name>.eastus.models.ai.azure.com/chat/completions
No api-version in mine, so I commented it out in the config.

[rwmehta]

Ended up being an issue with my endpoint, thanks for the help!

[amalthomasintel]

@rwmehta , Having similar problem.. How did you fix the issue?

[rwmehta]

You have to use the azure ai inference endpoint, which is of this format:
https://<resource-name>.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview.

The normal serverless endpoint doesn't work, I think it's API is expecting something different from what LibreChat is sending.