Chat Response Gets Stuck in Loading State with Azure OpenAI Models in a Semi-Production Environment

[genaiservice]

What happened?

Description:

I'm experiencing a bug in a semi-production environment when using the following deploy-compose.yml configuration for the application:

version: "3.8"
services:
  api:
    container_name: Ingeteam-Copilot
    ports:
      - "PORT:{PORT}"
    image: ingeteam/ingeteam-copilot:0.2.0.2
    restart: always
    extra_hosts:
      - "host.docker.internal:host-gateway"
    env_file:
      - .env
    environment:
      - HOST=0.0.0.0
      - NODE_ENV=production
      - MONGO_URI=${MONGO_URI}
    volumes:
      - type: bind
        source: ./.env
        target: /app/.env
      - ./images:/app/client/public/images
      - ./librechat.yaml:/app/librechat.yaml
  client:
    image: nginx:1.27.0-alpine
    container_name: ingeteam-copilot-NGINX
    ports:
      - 8080:80
      - 8443:443
    depends_on:
      - api
    restart: always
    volumes:
      - ./client/nginx.conf:/etc/nginx/conf.d/default.conf
      - ./ingeteamCopilotCerts:/etc/nginx/ssl

Issue Details:

When making a request using Azure OpenAI models, the message is generated correctly, but the application gets stuck in a loading state for about 30 seconds before completing the action. During this time, the application is unresponsive, and it eventually loads the response, but it's unclear why it takes those 30 seconds. Notably, when conducting tests on localhost, I do not encounter this issue.

Expected Behavior:

Upon sending a message, the application should respond quickly without getting stuck for an extended period. The chat should display the generated message and any subsequent response without delay.

Actual Behavior:

After sending a message, the chat remains in a loading state for about 30 seconds before finally ending the response from the model.

Additional Information:

The configurations for MongoDB and other environment variables are set correctly in the .env file.
All services appear to be running without any errors in the logs.
The issue does not occur when testing locally.
Please let me know if you need any other information to help resolve this issue. Thank you!

Version Information

The image I am using has been created by modifying the repository from the main branch. The modifications are primarily related to the frontend and are based on the following Dockerfile:

# v0.7.6

# Base node image
FROM node:20-alpine AS node

RUN apk --no-cache add curl

RUN mkdir -p /app && chown node:node /app
WORKDIR /app

# Install npm 11.1.0 globally
RUN npm install -g [email protected]

USER node

COPY --chown=node:node . .

RUN \
    # Allow mounting of these files, which have no default
    touch .env ; \
    # Create directories for the volumes to inherit the correct permissions
    mkdir -p /app/client/public/images /app/api/logs ; \
    npm config set fetch-retry-maxtimeout 600000 ; \
    npm config set fetch-retries 5 ; \
    npm config set fetch-retry-mintimeout 15000 ; \
    npm install --no-audit; \
    # React client build
    NODE_OPTIONS="--max-old-space-size=2048" npm run frontend; \
    npm prune --production; \
    npm cache clean --force

RUN mkdir -p /app/client/public/images /app/api/logs

# Node API setup
EXPOSE 3080
ENV HOST=0.0.0.0

CMD ["npm", "run", "backend"]

# Optional: for client with nginx routing
# FROM nginx:stable-alpine AS nginx-client
# WORKDIR /usr/share/nginx/html
# COPY --from=node /app/client/dist /usr/share/nginx/html
# COPY client/nginx.conf /etc/nginx/conf.d/default.conf
# ENTRYPOINT ["nginx", "-g", "daemon off;"]
```

### Steps to Reproduce

1. Build the docker image
2. Run the deploy-compose.yml
3. Chat with the model

### What browsers are you seeing the problem on?

Chrome

### Relevant log output

```shell

```

### Screenshots

Here is a screenshot that may help understand the issue.

![Image](https://github.com/user-attachments/assets/818cbdb1-bfed-4619-93b0-9c6270129853)

### Code of Conduct

- [x] I agree to follow this project's Code of Conduct

[danny-avila]

There is not enough information to help, could you inspect how your server is performing when running and the app is used?

In a remote VPS (1 vCPU, 2 GB RAM), I have no issues with Azure using the same compose file.

firefox_tobDnwppFA.mp4

[danny-avila]

Also worth inspecting the logs.

Log locations:

• Docker: Project root directory ./logs
• npm: ./api/logs

There are two types of logs that can help diagnose the issue:

• debug logs (debug-YYYY-MM-DD.log)
• error logs (error-YYYY-MM-DD.log)