[Question]: Configure 2 OpenAI groups with same models on different baseUrl in librechat.yaml

[schnaker85]

What is your question?

hi!

We are trying to configure librechat 0.7.2 with the following requirement:

• multiple azureOpenAI Endpoints on different baseUrl
• one for assistant and the other without
• same provided models

From my point of view this is not possible (also did not get it working with the attached configuration). As the models name must be unique (see documentation). I can either enable the first or the second group but not both. Also I do not see any chance to merge the configurations into the same group dude to different baseUrls/apikeys.

Is this something which somehow should work or how do I make both available in the LibreChat GUI?

Thanks a lot for your help!

Cheers
dani

More Details

Our librechat.yaml looks like the following:

    # For more information, see the Configuration Guide:
    # https://www.librechat.ai/docs/configuration/librechat_yaml
    
    # Configuration version (required)
    version: 1.0.9
    
    # Cache settings: Set to true to enable caching
    cache: true
    
    endpoints:
      # TODO: will soon be renamed to azureAssistants
      assistants:
        supportedIds: ["asst_yyyy"]
        capabilities: ["actions", "tools"]
        disableBuilder: true
        privateAssistants: false
      azureOpenAI:
        # Endpoint-level configuration
        titleModel: current_model
        plugins: false
        assistants: true
        groups:
        - group: "yyy-sweden"
          baseUrl: https://yyy-sweden.openai.azure.com/
          apiKey: "yyy"
          instanceName: "yyy-sweden"
          deploymentName: "gpt-4-1106-preview"
          version: "2024-02-15-preview"
          assistants: true
          models:
            gpt-4: true
        - group: "xxx"
          apiKey: "xxx"
          baseUrl: "https://xxx-north.openai.azure.com/"
          instanceName: "xxx-north"
          deploymentName: gpt-4-32k
          version: "2023-07-01-preview"
          assistants: false
          plugins: true
          models:
            gpt-4: true.   # this breaks
            

    rateLimits:
      fileUploads:
        ipMax: 100 # Maximum number of uploads allowed per IP address per window.
        ipWindowInMinutes: 60  # Rate limit window for file uploads per IP
        userMax: 50 # Maximum number of uploads allowed per user per window.	          
        userWindowInMinutes: 60  # Rate limit window for file uploads per user
      
    fileConfig:
      endpoints:
        default:
          totalSizeLimit: 5  # The total maximum size allowed for all files in a single request, specified in megabytes (MB).
          supportedMimeTypes:
            - "image/.*"
            - "application/pdf"
        google: 
          disabled: false
        openAI:
          disabled: false
      serverFileSizeLimit: 10  # Global server file size limit in MB
      avatarSizeLimit: 2  # Limit for user avatar image size in MB

What is the main subject of your question?

Installation, Endpoints

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

It’s possible, just add an arbitrary name after the main identifier I.e “gpt-4-asst”

[schnaker85]

Hi,

I will give it a try and let you know if it works 👍

Thanks!

[schnaker85]

It works as you suggested. The only drawback is that now in the OpenAI model dropdown also the gpt-4-asst is present. Is there a way to hide models, which should only be used in assistants?

[danny-avila]

At the moment, no, but I can try to see how to implement this simple use case.