Getting error while configuring librechat with azure openAI

[Chethanibms]

We are getting following errors on configuring librechat with azure openAI, Can someone help with this.

LibreChat-API     | 2024-03-27 06:19:39 warn: The `PLUGINS_USE_AZURE` environment variable (related to using Azure for Plugins) should not be used in combination with the `azureOpenAI` endpoint configuration, as you will experience conflicts and errors.
LibreChat-API     | 2024-03-27 06:19:52 error: Failed to fetch models from OpenAI API Something happened in setting up the request Cannot read properties of undefined (reading 'status')
LibreChat-API     | 2024-03-27 06:19:54 error: [MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index bc68285c-086c-4ff1-b666-21072d3d4dbc Document `bc68285c-086c-4ff1-b666-21072d3d4dbc` not found.
LibreChat-API     | 2024-03-27 06:19:55 warn: [OpenAIClient.chatCompletion][stream] API error
LibreChat-API     | 2024-03-27 06:19:55 error: [handleAbortError] AI response error; aborting request: Connection error.

[Chethanibms]

Our configuration file will like this.
We are using the latest version

# For more information, see the Configuration Guide:
# https://docs.librechat.ai/install/configuration/custom_config.html

# Configuration version (required)
version: 1.0.5

# Cache settings: Set to true to enable caching
cache: true

# Custom nterface configuration
interface:
  # Privacy policy settings
  privacyPolicy:
    externalUrl: 'https://librechat.ai/privacy-policy'
    openNewTab: true

  # Terms of service
  termsOfService:
    externalUrl: 'https://librechat.ai/tos'
    openNewTab: true

# Example Registration Object Structure (optional)
registration:
  socialLogins: ['github', 'google', 'discord', 'openid', 'facebook']
  # allowedDomains:
  # - "gmail.com"
# rateLimits:
#   fileUploads:
#     ipMax: 100
#     ipWindowInMinutes: 60  # Rate limit window for file uploads per IP
#     userMax: 50
#     userWindowInMinutes: 60  # Rate limit window for file uploads per user

# Definition of custom endpoints
endpoints:
  azureOpenAI:
    titleModel: "gpt-4-turbo"
    plugins: true
    groups:
    - group: "test" # arbitrary name
      apiKey: "xxxxx"
      instanceName: "xxxxx" # name of the resource group or instance
      version: "vision-preview"
      models:
        gpt-4:
          deploymentName: gpt-4
          version: "vision-preview"

[danny-avila]

Did you see the error log in your original comment?

The PLUGINS_USE_AZURE environment variable (related to using Azure for Plugins) should not be used in combination with the azureOpenAI endpoint configuration, as you will experience conflicts and errors.

[Chethanibms]

Is it possible to deploy Librechat using azureOpenAI.

[danny-avila]

Of course but we need more context to help. When is the error happening? What does the app look like when it does? What are your logs when it happens (from ./logs if using docker, ./api/logs if running locally, from project directory)?

With a proper setup, I can use all of these models (each representing a different deployment, some on different regions) seamlessly.

My setup:

version: 1.0.5
cache: true
endpoints:
  azureOpenAI:
    titleModel: "gpt-3.5-turbo-1106"
    titleConvo: true
    plugins: true
    groups:
    - group: "region-westus"
      apiKey: "${WESTUS_API_KEY}"
      instanceName: "region-westus"
      version: "2024-03-01-preview"
      models:
        gpt-3.5-turbo:
          deploymentName: "gpt-35-turbo"
        gpt-3.5-turbo-1106:
          deploymentName: "gpt-35-turbo-1106"
        gpt-4:
          deploymentName: "gpt-4"
    - group: "region-sweden"
      apiKey: "${SWEDEN_API_KEY}"
      instanceName: "region-sweden"
      deploymentName: gpt-4-1106-preview
      version: "2024-03-01-preview"
      models:
        gpt-4-turbo: true
    - group: "azure-openai"
      apiKey: "${AZURE_OAI_API_KEY}"
      instanceName: "azure-openai"
      deploymentName: gpt-4-1106-preview
      version: "2024-03-01-preview"
      models:
        gpt-4-1106-preview: true
        gpt-4-vision-preview:
          deploymentName: gpt-4-vision-preview

[askxue]

When I define azure endpoint directly in librechat.yaml, I get an error

Something went wrong. Here's the specific error message we encountered: Either a model name with the AZURE_USE_MODEL_AS_DEPLOYMENT_NAME setting or a deployment name must be provided if AZURE_OPENAI_BASEURL is omitted.

So I defined it in.env

`AZURE_OPENAI_BASEURL=https://${AZURE_OPENAI_API_INSTANCE_NAME}.openai.azure.com/openai/deployments/${AZURE_OPENAI_API_D EPLOYMENT_NAME} ',

Then an error will be reported

Something went wrong. Here's the specific error message we encountered: 404 Resource not found

So I defined all the azure related variables directly in.env, and the result was fine. I don't know what went wrong.

[Chethanibms]

Hi @danny-avila

How do we need to pass azure openAI env's when installing Librechat using helmchart.

[askxue]

I'm a newbie and new to librechat.

I also encountered the same problem.

.env

AZURE_API_KEY=xxxxxx
AZURE_OPENAI_BASEURL=https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}

librechat.yaml

version: 1.0.5
cache: true

interface:
  privacyPolicy:
    externalUrl: 'https://librechat.ai/privacy-policy'
    openNewTab: true

  termsOfService:
    externalUrl: 'https://librechat.ai/tos'
    openNewTab: true

registration:
  socialLogins: [ 'github', 'google', 'discord', 'openid', 'facebook' ]

endpoints:
  azureOpenAI:
    titleModel: "gpt-4-0125-preview"
    plugins: true
    summarize: false
    titleConvo: false
    titleMethod: "completion"
    groups:
      - group: "my-resource"
        apiKey: "${AZURE_API_KEY}"
        instanceName: "xxx"
        version: "2024-03-01-preview"
        models:
          gpt-4-0125-preview:
            deploymentName: "gpt-4"
  assistants:
    capabilities: [ "code_interpreter", "actions", "tools" ]

By the way, I'm in China, I don't know if it has an impact.😢

[peng2x]

For me, the below error was caused by using the wrong Azure OpenAPI Version.

Something went wrong. Here's the specific error message we encountered: 404 Resource not found

Correct librechat.yaml:

    groups:
    - group: "my-group"
      apiKey: mykey
      instanceName: "myinstance"
      deploymentName: gpt-4o-mini
      version: "2023-03-15-preview" #  Correct api-version from https://myinstance.openai.azure.com/openai/deployments/gpt-4o-mini/chat/completions?api-version=2023-03-15-preview
      assistants: true
      models:
        gpt-4o-mini: true

Wrong librechat.yaml:

    groups:
    - group: "my-group"
      apiKey: mykey
      instanceName: "instance"
      deploymentName: gpt-4o-mini
      version: "2024-07-18" #  Incorrectly using the gpt-4o-mini model version
      assistants: true
      models:
        gpt-4o-mini: true

After correcting the version in librechat.yaml, it works without defining the Azure OpenAI variables in the .env.

[aadishj19]

This is an old thread, but just wanted to acknowledge that this helped me. You can get the correct API version from https://myinstance.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview