Seeking Community Feedback: Security Considerations for LibreChat Deployment

[jainrahulsethi]

Hi Everyone,

I’m planning to deploy LibreChat with an externally hosted LLM while self-hosting all other components within a single VM. As this will be used by internal business users, I'm focusing heavily on security aspects, especially since users might upload sensitive data.

I would love to get your expert advice and feedback on a few checklist items, and any additional considerations you think I should address before going live:

1. Egress Traffic:
   Beyond the LLM endpoint, is there any other expected egress traffic from this app?

2. Encryption at Rest:
   How can we enable encryption for all data stored locally on the VM? Any recommendations for lightweight but robust solutions?

3. File and Embedding Storage:

   • Are uploaded files stored somewhere, or only the embeddings?
   • How can we configure auto-deletion policies for stored data?

4. PII Data Masking:
   Any suggestions for masking or anonymizing personally identifiable information (PII) before storing or processing?

5. Log Retention Policies:

   • How should we handle logging securely?
   • Are there configurations to control retention and prevent sensitive data exposure?

6. Terms & Conditions:
   Are there any explicit terms and conditions from LibreChat or its components that I should be aware of for compliance or legal purposes?

7. Compliance (GDPR, etc.):
   If used internally for business purposes, do frameworks like GDPR or others apply?

8. Ingress Traffic Control:
   I’m considering limiting ingress traffic via a VPC—any best practices for such setups?

9. Anything else which should be specifically considered?

I truly appreciate any insights, feedback, or experiences you can share. Your expertise will help ensure a secure and compliant deployment for this use case.

Looking forward to learning from the community—thank you in advance!

[tmccoy]

Hi @jainrahulsethi I'm just getting started with LibreChat and I have many similar questions! I'm still quite new but I'll try and answer some of these q's. I am not associated with the project at all.

1. I think it's just the LLM traffic for self hosted, keep in mind any MCP you setup will also egress traffic off the VM. Such as if you setup MCP for accessing jira or something. Also, if you setup embeddings, data will be sent to the embedding provider.

2. You'll just have to enable this using whatever your provider provides. If you're using AWS or some other cloud there are options for at rest encryption for the disk when you setup the VM.

3. Looking at the code, it looks like uploaded files are temporarily stored on the librechat api host, then uploaded to mongo and uploaded to the embedding provider and then removed from the librechat api host. Files uploaded to mongo are not removed that I can see. I might push up a PR to delete file attachments when a conversation deleted assuming I haven't missed something here. There is no way to configure auto deletion of data that I'm aware of.

4. I'm not aware of anything at the moment, good question.

5. Logging is completely managed by you when you deploy so this is really just up to how you're deploying/managing librechat.

6. The license for the opensource project is here: https://github.com/danny-avila/LibreChat/blob/main/LICENSE and if you are using anything on librechat.ai the tos is here: https://www.librechat.ai/tos

7. This is a question for your security/compliance team at your company.

8. This is hard to answer without understanding your platform. Assuming you mean like AWS, yea you can restrict access to the VM, load balancer or reverse proxy in front of the deployment to only traffic from a particular VPC. This is more of a provider question than a librechat question.

[jainrahulsethi]

Thanks for your take. Appreciate it

[tmccoy]

Np, did you end up getting deployed in your business?