CONNREFUSED even though LiteLLM on the same Docker network

[whalygood]

Still banging my head against the wall on this one. Used to have this working but not in the latest version.
Any glaring issues with the configuration below?
 
My LiteLLM is running on the same Docker network (librechat_default) as my LibreChat app but am getting this error message when chatting with the Bedrock model through LibreChat.

{"cause":{"code":"ECONNREFUSED","errno":"ECONNREFUSED","message":"request to http://librechat-litellm-1:4000/chat/completions failed, reason: connect ECONNREFUSED 172.18.0.8:4000","type":"system"},"level":"error","message":"[handleAbortError] AI response error; aborting request: Connection error.","stack":"Error: Connection error.\n at OpenAI.makeRequest

In my librechat.yaml the LiteLLM proxy container is configured as follows

endpoints:

  custom:
    - name: "Amazon Bedrock"
      apiKey: "sk-1234"
      baseURL: "http://librechat-litellm-1:4000"
      models:
        default: ["Llama 3.1 Instruct 70B","Claude 3.5 Sonnet"]
        fetch: true
      titleConvo: true
      titleModel: "Llama 3.1 Instruct 70B"
      summarize: false
      summaryModel: "Llama 3.1 Instruct 70B"
      forcePrompt: false
      modelDisplayLabel: "Amazon Bedrock"

I can see from inspecting the Docker network docker inspect librechat_default that both the api and librechat-litellm-1 containers are on the same network.

                "Name": "LibreChat-API",
                "EndpointID": "c651e62fae72dc0ce2c4960fb65be9cda6087f0a286b075347b3c8969bb258de",
                "MacAddress": "02:42:ac:12:00:06",
                "IPv4Address": "172.18.0.6/16",
                "IPv6Address": ""
            },
            "f32242ace6bf928b53cc96b8da23bddde86a5c0f1a0136d81ca0253244542d6c": {
                "Name": "librechat-litellm-1",
                "EndpointID": "c41cd544f6145ae724271ac1332d5b609436f4b1dcc3549d2de98e9151ace597",
                "MacAddress": "02:42:ac:12:00:08",
                "IPv4Address": "172.18.0.8/16",
                "IPv6Address": ""
            }
        },

It also doesn't work if I specify baseURL: "http://litellm:4000". Same error message

The curl command to the LiteLLM directly does work.

$ curl --location 'http://localhost:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer sk-1234" \
--data ' {
      "model": "Claude 3.5 Sonnet",
      "messages": [
        {
          "role": "user",
          "content": "what llm are you"
        }
      ]
    }
'

If it makes a difference I am using deploy-compose.yml instead of docker-compose.yml and passing both the deploy file and the override file using the -f flags to the docker compose up -d command

[zacharycavanaugh]

I'm having this exact issue as well.

I have no issues with litellm-proxy using docker-compose.yml locally but the same or similar configurations aren't working with docker-deploy.yml.