[Question]: How can I add Mistral to the list of endpoints?

[berceanu]

What is your question?

I have recently obtained an API key for Mistral and would like to use it through LibreChat. However, I do not see Mistral in the drop-down list (screenshot below), even though I added said API key to librechat.yaml file and ran docker-compose build --no-cache afterwards. Using LibreChat v 0.6.5.

More Details

# Configuration version (required)
version: 1.0.0

# Cache settings: Set to true to enable caching
cache: true

# Definition of custom endpoints
endpoints:
  custom:
    # Mistral AI API
    - name: "Mistral"  # Unique name for the endpoint
      # For `apiKey` and `baseURL`, you can use environment variables that you define.
      # recommended environment variables:
      apiKey: "*********************************"
      baseURL: "https://api.mistral.ai/v1"

      # Models configuration
      models:
        # List of default models to use. At least one value is required.
        default: ["mistral-tiny", "mistral-small", "mistral-medium"]
        # Fetch option: Set to true to fetch models from API.
        fetch: true  # Defaults to false.

      # Optional configurations

      # Title Conversation setting
      titleConvo: true  # Set to true to enable title conversation

      # Title Method: Choose between "completion" or "functions".
      titleMethod: "completion"  # Defaults to "completion" if omitted.

      # Title Model: Specify the model to use for titles.
      titleModel: "mistral-tiny"  # Defaults to "gpt-3.5-turbo" if omitted.

      # Summarize setting: Set to true to enable summarization.
      summarize: false

      # Summary Model: Specify the model to use if summarization is enabled.
      summaryModel: "mistral-tiny"  # Defaults to "gpt-3.5-turbo" if omitted.

      # Force Prompt setting: If true, sends a `prompt` parameter instead of `messages`.
      forcePrompt: false

      # The label displayed for the AI model in messages.
      modelDisplayLabel: "Mistral"  # Default is "AI" when not set.

      # Add additional parameters to the request. Default params will be overwritten.
      addParams:
        safe_mode: true # This field is specific to Mistral AI: https://docs.mistral.ai/api/

      # Drop Default params parameters from the request. See default params in guide linked below.
      dropParams: ["stop", "temperature", "top_p"]
      # - stop # dropped since it's not recognized by Mistral AI API
      # `temperature` and `top_p` are removed to allow Mistral AI API defaults to be used:
      # - temperature
      # - top_p

    # OpenRouter.ai Example
    - name: "OpenRouter"
      # For `apiKey` and `baseURL`, you can use environment variables that you define.
      # recommended environment variables:
      # Known issue: you should not use `OPENROUTER_API_KEY` as it will then override the `openAI` endpoint to use OpenRouter as well.
      apiKey: "${OPENROUTER_KEY}"
      baseURL: "https://openrouter.ai/api/v1"
      models:
        default: ["gpt-3.5-turbo"]
        fetch: true
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: false
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "OpenRouter"

# See the Custom Configuration Guide for more information:
# https://docs.librechat.ai/install/configuration/custom_config.html

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Hi @berceanu Thanks for checking out the project! I'm happy to say it will work for you.

For Docker, there is an extra step to have it configured: https://docs.librechat.ai/install/configuration/custom_config.html#docker-setup

I will copy/paste the instructions below.

Using LibreChat v 0.6.5.

Make sure you are on the latest commit.

    safe_mode: true # This field is specific to Mistral AI: https://docs.mistral.ai/api/

I noticed this field in your config. The example has been updated to remove this as Mistral API has deprecated safe_mode in favor of safe_prompt and it will throw an error if you use the former. Once on the latest commit, you can use the most up to date librechat.example.yaml. I will also include it below

---

Docker Setup

For Docker, you need to make use of an override file, named docker-compose.override.yml, to ensure the config file works for you.

• First, make sure your containers stop running with docker-compose down
• Create or edit existing docker-compose.override.yml at the root of the project:

# For more details on the override file, see the Docker Override Guide:
# https://docs.librechat.ai/install/configuration/docker_override.html

version: '3.4'

services:
  api:
    volumes:
      - ./librechat.yaml:/app/librechat.yaml

• Start docker again, and you should see your config file settings apply

docker-compose up # no need to rebuild

---

Updated example

# Configuration version (required)
version: 1.0.1

# Cache settings: Set to true to enable caching
cache: true

# Definition of custom endpoints
endpoints:
  custom:
    # Mistral AI API
    - name: "Mistral"  # Unique name for the endpoint
      # For `apiKey` and `baseURL`, you can use environment variables that you define.
      # recommended environment variables:
      apiKey: "${MISTRAL_API_KEY}"
      baseURL: "https://api.mistral.ai/v1"

      # Models configuration
      models: 
        # List of default models to use. At least one value is required.
        default: ["mistral-tiny", "mistral-small", "mistral-medium"]
        # Fetch option: Set to true to fetch models from API.
        fetch: true  # Defaults to false.

      # Optional configurations
      
      # Title Conversation setting
      titleConvo: true  # Set to true to enable title conversation

      # Title Method: Choose between "completion" or "functions".
      titleMethod: "completion"  # Defaults to "completion" if omitted.

      # Title Model: Specify the model to use for titles.
      titleModel: "mistral-tiny"  # Defaults to "gpt-3.5-turbo" if omitted.

      # Summarize setting: Set to true to enable summarization.
      summarize: false

      # Summary Model: Specify the model to use if summarization is enabled.
      summaryModel: "mistral-tiny"  # Defaults to "gpt-3.5-turbo" if omitted.

      # Force Prompt setting: If true, sends a `prompt` parameter instead of `messages`.
      forcePrompt: false

      # The label displayed for the AI model in messages.
      modelDisplayLabel: "Mistral"  # Default is "AI" when not set.

      # Add additional parameters to the request. Default params will be overwritten.
      addParams:
        safe_prompt: true # This field is specific to Mistral AI: https://docs.mistral.ai/api/
        
      # Drop Default params parameters from the request. See default params in guide linked below.
      # NOTE: For Mistral, it is necessary to drop the following parameters or you will encounter a 422 Error:
      dropParams: ["stop", "user", "frequency_penalty", "presence_penalty"]
      
    # OpenRouter.ai Example
    - name: "OpenRouter"
      # For `apiKey` and `baseURL`, you can use environment variables that you define.
      # recommended environment variables:
      # Known issue: you should not use `OPENROUTER_API_KEY` as it will then override the `openAI` endpoint to use OpenRouter as well.
      apiKey: "${OPENROUTER_KEY}"
      baseURL: "https://openrouter.ai/api/v1"
      models:
        default: ["gpt-3.5-turbo"]
        fetch: true
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: false
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "OpenRouter"

# See the Custom Configuration Guide for more information:
# https://docs.librechat.ai/install/configuration/custom_config.html

[berceanu]

Awesome, thanks a lot! Just tested and it works! 🥳