Lack of Streaming Response for Azure o1-mini Model

[genaiservice]

What happened?

What happened?

When using the o1-mini AzureOpenAI model in LibreChat, the model generates correct responses; however, the streaming functionality does not work as expected. With other models, streaming is functional and responses are displayed progressively (token by token). This inconsistency appears to be specific to the o1-mini model, despite the proper configuration being added in the librechat.yml file.

The following configuration is being used for the AzureOpenAI o1-mini model in the librechat.yml file:

endpoints:
  azureOpenAI:
    titleModel: "gpt-4o-mini"
    plugins: true
    assistants: false
    titleConvo: true
    titleMethod: "functions"
    groups:
      - group: "ingeteam-azure-models-gpt-4o1-mini"
        apiKey: '${AZURE_IRI_GENAI_O1_MINI_API_KEY}'
        instanceName: "gaizk-m7byyve7-eastus2"
        deploymentName: o1-mini-ingeteam-copilot
        version: "2024-08-01-preview"
        assistants: true
        models:
          o1-mini: true

Expected result:

When using the o1-mini model, the response should stream progressively, token by token, in the same way other supported models behave. This would provide a smoother and more interactive user experience.

Version Information

this is the docker-compose im currently using:

# Do not edit this file directly. Use a ‘docker-compose.override.yaml’ file if you can.
# Refer to `docker-compose.override.yaml.example’ for some sample configurations.

services:
  api:
    container_name: LibreChat
    ports:
      - "${PORT}:${PORT}"
    depends_on:
      - mongodb
      - rag_api
    image: ghcr.io/danny-avila/librechat-dev:latest
    restart: always
    user: "${UID}:${GID}"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      - HOST=0.0.0.0
      - MONGO_URI=mongodb://mongodb:27017/LibreChat
      - MEILI_HOST=http://meilisearch:7700
      - RAG_PORT=${RAG_PORT:-8000}
      - RAG_API_URL=http://rag_api:${RAG_PORT:-8000}
    volumes:
      - type: bind
        source: ./.env
        target: /app/.env
      - ./images:/app/client/public/images
      - ./uploads:/app/uploads
      - ./logs:/app/api/logs
      - ./librechat.yaml:/app/librechat.yaml
  mongodb:
    container_name: chat-mongodb
    image: mongo
    restart: always
    user: "${UID}:${GID}"
    volumes:
      - ./data-node:/data/db
    command: mongod --noauth
  meilisearch:
    container_name: chat-meilisearch
    image: getmeili/meilisearch:v1.12.3
    restart: always
    user: "${UID}:${GID}"
    environment:
      - MEILI_HOST=http://meilisearch:7700
      - MEILI_NO_ANALYTICS=true
      - MEILI_MASTER_KEY=${MEILI_MASTER_KEY}
    volumes:
      - ./meili_data_v1.12:/meili_data
  vectordb:
    container_name: vectordb
    image: ankane/pgvector:latest
    environment:
      POSTGRES_DB: mydatabase
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
    restart: always
    volumes:
      - pgdata2:/var/lib/postgresql/data
  rag_api:
    container_name: rag_api
    image: ghcr.io/danny-avila/librechat-rag-api-dev-lite:latest
    environment:
      - DB_HOST=vectordb
      - RAG_PORT=${RAG_PORT:-8000}
    restart: always
    depends_on:
      - vectordb
    env_file:
      - .env

volumes:
  pgdata2:

Steps to Reproduce

• 1- Clone the LibreChat repository: git clone https://github.com/danny-avila/LibreChat.git
• 2- Add the provided AzureOpenAI o1-mini model configuration to the librechat.yml file, modify the docker-compose to the one provided in the version section and update the .env file with your Azure API key. Ensure the AzureOpenAI endpoint is enabled.
• 3- Run the application using Docker: docker compose up
• 4- Interact with the o1-mini model through the LibreChat UI and observe the generated responses. Note that no streaming occurs, and the response is displayed only after the entire completion of the generation.

What browsers are you seeing the problem on?

Chrome

Relevant log output

Screenshots

Grabando-20250304_165452.webm

Code of Conduct

• I agree to follow this project's Code of Conduct

[KiGamji]

same for o1-preview and o1

[danny-avila]

See: Azure/azure-sdk-for-net#47663

[danny-avila]

and https://community.openai.com/t/do-we-also-have-streaming-support-for-azure-openai-for-o1-preview/1035400