[Question]: How can i get this running on AWS via docker

[noobskies]

Contact Details

[email protected]

What is your question?

how exactly can I expose this with docker on aws?

I have changed the following

version: "3.4"

services:
  client:
    image: nginx-client
    build:
      context: .
      target: nginx-client
    restart: always
    ports:
      - 3080:80
    volumes:
      - /client/node_modules
    depends_on:
      - api
  api:
    container_name: chat-clone
    ports:
      - 9000:3080         

what else do I need changed?

More Details

do I need to change anything in the .env?

What is the main subject of your question?

Installation

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

I have not attempted deploying on AWS. Some people have said they had issues doing so. You may also need to change the env vars set in each env file as well as the compose file. You're mostly on your own for this, you can ask around in the discord

[danorlando]

This will require a good amount of experimenting. I've attempted to get the project running several times using AWS Elastic Container Service and Elastic Beanstalk but haven't had any luck so far. It always fails at the start script with: There was an uncaught error: getaddrinfo ENOTFOUND 0.0.0.0

I haven't figured out how to modify the configuration to get a different result. Please let us know if you are able to make progress with it. It would be greatly appreciated.

[noobskies]

I actually got it working https://gptchina.io/ although this was more POC than anything, this is just on an ec2 barebones + standalone nginx

[noobskies]

semi straightforward, ecr is really just the registry so it's not that much diff

[noobskies]

never really contributed before either ha but I could get you something by tomorrow most likely if that's what you're keen on, really love the project

[danny-avila]

@noobskies 🙏 that would be awesome, please contribute and glad the project has been helpful!

[BustosAndrew]

@noobskies did you come up with the steps with deploying on ec2 yet?

[renatosaz]

@noobskies and....as you said yourself, never contributed , and probably never will 👯

[danny-avila]

@BustosAndrew @dralucindaodonto I’m fairly confident you can get it on an EC2 instance following 90% of the guide here: https://docs.librechat.ai/deployment/digitalocean.html

the only thing that would be missing is AWS-specific information, like getting your SSH keys and using their interface, etc.

also I recommend digitalocean over AWS as I can see it being cheaper especially with the credit

[noobskies]

hey @BustosAndrew, I'll get to this tomorrow sorry irl happened

[badcom]

Has anyone figured out how to deploy this to AWS using ECR? @noobskies ?

[sudheermanubolu]

I have publish a CDK to deploy LibreChat on AWS using ECS and AWS managed databases. No hardcoded AWS creds, solution will leverage AWS IAM to connect to other AWS services.

https://github.com/sudheermanubolu/librechat-cdk

[epearson]

@davyraittdevelops I'm trying to remember everything I did when I first got it going ... I was getting that same "RAG API is either not running" error. Looking back in my old logs I also see an error where the Model was unreachable which may have also caused problems. Try looking in your librechat.env config EMBEDDINGS_PROVIDER and EMBEDDINGS_MODEL. I commented out the EMBEDDINGS_MODEL and changed EMBEDDINGS_PROVIDER=openai. If you prefer the bedrock models, I'd check your logs to see if there was an error. I had made an adjustment in the CDK to the AWS Bedrock iam permissions.

[davyraittdevelops]

Thanks sir, already checked also. Have setup like this:

EMBEDDINGS_PROVIDER=bedrock
EMBEDDINGS_MODEL=amazon.titan-embed-g1-text-02
AWS_REGION=us-east-1

Currently set up bedrock:* to * for debugging, will move to least privilege later on but wanted to exclude any permission issues.
Any tips on where to look for logs? In the ECR logs itself i dont see much, I can maybe remote into the ECS task but not sure where to look there. Thanks for the tips/help by the way, also debugging multiple things atm to see what can possibly be the issue. So might figure it out on my own eventually : )

[epearson]

I only know about the ECR logs. I would think there'd be some clue in either the LibreChat service or the RAG API service logs that would give a clue. My deployment is in a completely scratch AWS account and allowing the cdk to generate the vpc, etc.

[sudheermanubolu]

Can you check the ECS task logs for RAG service and can you confirm if you have access to the embeddings model you are using, in bedrock? ECS task logs should give some insights.

[davyraittdevelops]

This is the error I see in the console

`{
"message": "Request failed with status code 403",
"name": "AxiosError",

1. "stack": "AxiosError: Request failed with status code 403\n    at hb (https://xxx.poc.company.nl/assets/vendor.BQhDEHht.js:50:102204)\n    at XMLHttpRequest.g (https://xxx.poc.company.nl/assets/vendor.BQhDEHht.js:50:107189)\n    at Hb.request (https://xxx.poc.company.nl/assets/vendor.BQhDEHht.js:50:115369)",
   

   "config": {
   "transitional": {
   "silentJSONParsing": true,
   "forcedJSONParsing": true,
   "clarifyTimeoutError": false
   },
   "adapter": [
   "xhr",
   "http",
   "fetch"
   ],
   "transformRequest": [
   null
   ],
   "transformResponse": [
   null
   ],
   "timeout": 0,
   "xsrfCookieName": "XSRF-TOKEN",
   "xsrfHeaderName": "X-XSRF-TOKEN",
   "maxContentLength": -1,
   "maxBodyLength": -1,
   "env": {},
   "headers": {
   "Accept": "application/json, text/plain, /",
   "Content-Type": "multipart/form-data",
   "Authorization": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjY4MDhjN2ZkY2NkZTBkYmE4N2JmN2NhOCIsInVzZXJuYW1lIjoiZGF2eS5yYWl0dEBlc3NlbnQubmwiLCJwcm92aWRlciI6ImxvY2FsIiwiZW1haWwiOiJkYXZ5LnJhaXR0QGVzc2VudC5ubCIsImlhdCI6MTc0NTQwOTQwMywiZXhwIjoxNzQ1NDEwMzAzfQ.mXqnWJJ3eJEZ-VsFKdL_whEeqR9ZswzaPur7AbqLISA"
   },
   "signal": {},
   "method": "post",
   "url": "/api/files",
   "data": {}
   },
   "code": "ERR_BAD_REQUEST",
   "status": 403
   }`

And I dont see any errors in the ECS logs, all looks good. Im going to try and remote into the ecs and see if i can get more info there