ChatLocalAI (GPT4ALL) Request failed with status code 500

[hannahnguyen14]

I am trying to use ChatLocalAI as the chat (language) model but it keeps returning the error of "Request failed with status code 500".

I don't know what is the issue here because I have followed the ChatLocalAI setup tutorial (https://docs.flowiseai.com/chat-models/chatlocalai). I have done until the Test API part before setup chat flows by running this command "curl http://localhost:8080/v1/models" and it returns successfully like the guide. I don't know what to expect when open http://localhost:8080 on google chrome tab, I have tried to and it returns "{"error":{"code":404,"message":"Cannot GET /","type":""}}".

Moreover, I have tried ChatLocalAI with 3 different chat flows like below and they return the same error. The chat flow with OpenAI embedding and pinecone vector database works fine with other chat models like OpenAI and Azure as I have tried.

Also, I have tried to change the base path to http://localhost:8080/ or add v1/models and it returns "failed with status code 404" so these paths seem wrong. Changing it to http://localhost:8080 base path returns the same error of 500.

I have tried my best, please help me out!

[chungyau97]

Hi @hannahnguyen14,

Is your Flowise cloud hosted or run locally?

[chungyau97]

The LocalAI URL: http://host.docker.internal:8080/v1
Pinecone Dimensions: 384

[hannahnguyen14]

Hi, I got trouble with base path http://host.docker.internal:8080/v1 as it returns this error: getaddrinfo ENOTFOUND host.docker.internal. Also, my pinecone dimension is 1536 instead, which I don't know if it cause any problems.

[chungyau97]

Troubleshoot steps:

1. Check Flowise and LocalAI use my scripts above to spin up; As my scripts above allow connection between 2 containers.
   docker-compose up -d
2. Both Flowise and LocalAI are running on the docker container
   Windows Docker UI:
   
   Linux: docker ps or sudo docker ps
   
3. Check LocalAI's models exists in Docker
   
4. Pinecone's dimensions need to be 384. If you run into errors using Pinecone, please upgrade your Pinecone to a paid version.

[hannahnguyen14]

Oh, I see what's the problem here now. Sorry for the confusion, I set up Flowise running quick start before but I tried to set up LocalAI using docker. Is there any way I can run LocalAI using 'npx' too?

Sorry for wasting your time, but I'm very positive about the process now as I can understand the problems underneath!

[HenryHengZJ]

might be related to this - #607

[matsuobasho]

My bad, I meant to type 8080.

1. Modify `LocalAI\docker-compose.yaml`

version: '3.6'

networks:
  net:
    driver: bridge

services:
  api:
    image: quay.io/go-skynet/local-ai:latest
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 8080:8080
    env_file:
      - .env
    volumes:
      - ./models:/models:cached
    networks:
      - net
    command: [ "/usr/bin/local-ai" ]

2. Build `LocalAI` and test following [ChatLocalAI](https://docs.flowiseai.com/chat-models/chatlocalai) and [LocalAI Embeddings](https://docs.flowiseai.com/embeddings/localai-embeddings)

3. Modify `Flowise\docker\docker-compose.yml`

version: '3.1'

services:
    flowise:
        image: flowiseai/flowise
        restart: always
        environment:
            - PORT=${PORT}
            - FLOWISE_USERNAME=${FLOWISE_USERNAME}
            - FLOWISE_PASSWORD=${FLOWISE_PASSWORD}
            - DEBUG=${DEBUG}
            - DATABASE_PATH=${DATABASE_PATH}
            - APIKEY_PATH=${APIKEY_PATH}
            - LOG_PATH=${LOG_PATH}
            - LOG_LEVEL=${LOG_LEVEL}
            - EXECUTION_MODE=${EXECUTION_MODE}
        ports:
            - '${PORT}:${PORT}'
        volumes:
            - ~/.flowise:/root/.flowise
        networks:
            - flowise_net
        command: /bin/sh -c "sleep 3; flowise start"

networks:
    flowise_net:
        name: localai_net
        external: true

4. Build Flowise following [Docker Compose](https://github.com/FlowiseAI/Flowise#docker-compose)

Does step 3 mean that I need to install flowise using Git and/or the Docker Compose method? Like the OP, I just installed using the quick start and so I only have the LocalAI instance in Docker, but not the Flowise instance. According to this discussion, it seems that to run LocalAi open models, I need to go down this route.

Additionally, if I need to use the Docker Compose method, then does that come after I perform the Git operations so that I can then add .env file?

[chungyau97]

Hi @matsuobasho,

Step 3 mean you will need to git clone Flowise repo and go through Docker Compose method. Reference: Docker Compose README.md

If you're using these methods [Quick Start, Developers], you do not have to follow steps shown above.

[matsuobasho]

Just to be clear, in order to run ChatLocalAI do I need to have both LocalAI AND Flowise running in the Docker container? From your guidance above, that seems to be the case. I installed using the Quick Start method and only have LocalAI instance in Docker, not Flowise.

So to go through the Docker Compose method, do I need to have the referenced file-structure, meaning that I need to install using the Developers approach (cloning the repo)?

Currently, I don't know which directory to add the .env file to. Here are the contents of my Flowise directory in Windows under C/Users/Uname/AppData/Roaming/npm/node_modules/flowise/

Thanks!

[chungyau97]

Hi @matsuobasho,

Step 3 mean you will need to git clone Flowise repo and go through Docker Compose method. Reference: Docker Compose README.md

If you're using these methods [Quick Start, Developers], you do not have to follow steps shown above.

Hey @matsuobasho,

If you're using these methods [Quick Start, Developers], you do not have to follow steps shown above.
This mean you do not need to have Flowise on Docker to run ChatLocalAI

In Flowise, we have a total of 4 installation methods [Quick Start, Developer, Docker Compose, Docker Image]. Each method required different methods of connecting to ChatLocalAI

1. Quick Start & Developer - Do not need to make Flowise into Docker in order to use ChatLocalAI
2. Docker Compose & Docker Image - Require modification of Flowise file tin order to use ChatLocalAI

[matsuobasho]

Thanks for the detailed explanation.
So is the following expected behavior then? I start LocalAI in Docker and then use the Quick Start method to launch Flowise.

I test the API, the following returns info with StatusCode 200:
curl http://localhost:8080/v1/models

I build a simple chatbot in Flowise and get a Request failed with status code 404 error when I ask the following question. Note I double-checked the spelling of the model name.

If I remove the http from the Base Path url, I get the following error:
connect ENETUNREACH 0.0.31.144:80

[chungyau97]

Hey @matsuobasho,

You got to use http://host.docker.internal:8080/v1

[matsuobasho]

Ok, thank you. Two things:

1. I think that link should be in the ChatLocalAI section of the tutorial (unless there's a way to get it to work with the original link http://localhost:8080/v1). Would have probably saved me more than a few hours. Happy to create a separate issue for that.
   

2. Why does it take such a long time to get a response? First question took 1.5 minutes and a follow-up question is still processing 17 minutes after. I realize this it outside the scope of this issue but grateful for any explanation or links! Is it just querying the LocalAI LMs through Docker takes a long time?

[chungyau97]

Hi @matsuobasho

1. It would be awesome if you could help us modified the docs.

• https://github.com/FlowiseAI/FlowiseDocs/blob/main/embeddings/localai-embeddings.md
• https://github.com/FlowiseAI/FlowiseDocs/blob/main/chat-models/chatlocalai.md

2. The inference process is slow on the CPU and any way to accelerate it? mudler/LocalAI#646

[matsuobasho]

@chungyau97 , apologies for delay in response.

1. Before I work on updating the docs with the correct path, please help me understand what I should update to. i.e., is the original base path (http://localhost:8080/v1) valid for some scenarios of running LocalAI/ Flowise? Or should I just replace it completely with the http://api:8080 path?
2. I have another workflow for asking questions about a document I load. I'm running both Flowise and LocalAI through Docker. As you can see below, I've specified the path mentioned in point 1. I'm using the text-embedding-ada-002.yaml embeddings that are in the same LocalAI models directory. And I get the same - Error request failed with status code 500.
   

As I mentioned here, using the api:8080 path does work with the simple ChatLocalAI + Prompt Template + LLM chain.

Thanks!

[chungyau97]

Hi @matsuobasho, it should be in http://host.docker.internal:8080/v1.