[Question]: Setup RAG on HuggingFace deployment

[lapeyrade]

What is your question?

Hello,

Thanks a lot for your amazing work! I have deployed an instance of LibreChat to HuggingFace with MongoDB Atlas and I would like to use RAG. I've set the RAG_OPENAI_API_KEY to my OpenAI key, but according to the log, I'm missing a RAG_API_URL. I'm not sure what I should do here.

More Details

I would like to know the simplest way to configure one.

Thank you very much!

What is the main subject of your question?

Installation

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[00sapo]

Have you managed to solve this problem?

[lapeyrade]

I didn't look into it further, sorry.

[fuegovic]

⚠️ Both your LibreChat and RAG_API Hugging Face Spaces must be public for this integration to work.

1. Set Up Your MongoDB Database

1. Use MongoDB Compass to connect to your LibreChat database.
2. Create a new collection named rag.
3. In the new rag collection:

• Navigate to the Indexes tab.

• Switch to the Search Indexes view.

• Click Create Atlas Search Index, then configure it as follows:

  • Name: vector_index

  • Type: Vector Search

  • Index Definition:

    {
      "fields": [
        {
          "numDimensions": 1536,
          "path": "embedding",
          "similarity": "cosine",
          "type": "vector"
        },
        {
          "path": "file_id",
          "type": "filter"
        }
      ]
    }

• Note: If you wish to use Hugging Face as the provider, you will need to change the numDimensions value in the index definition.

  • The default Hugging Face model uses 384 dimensions.
  • The intfloat/multilingual-e5-large-instruct model uses 1024.
  • You can set a custom model during next step by specifying the EMBEDDINGS_MODEL environment variable.

2. Set Up the RAG_API Space on Hugging Face

1. Duplicate the space: LibreChat/RAG_API
2. In the Space Settings:

• Set the environment variable EMBEDDINGS_PROVIDER to your provider of choice: either openai or huggingface.

3. In the Secrets tab:

• Add your MongoDB connection URI as ATLAS_MONGO_DB_URI.

• Depending on your embedding provider:

  • For Hugging Face, set HF_TOKEN.
  • For OpenAI, set RAG_OPENAI_API_KEY.

3. Set Up the LibreChat Space

1. In your LibreChat space settings, add a new secret:

   • Key: RAG_API_URL
   • Value: https://<your-hf-username>-rag-api.hf.space
     (Example: https://librechat-rag-api.hf.space)

[00sapo]

Thank you so much! I think this guide should be added to the docs. Just for completeness: mongodb.com shows a connection url like:

mongodb+srv://user:<db_password>@librechat-cluster.e4wg71n.mongodb.net/?retryWrites=true&w=majority&appName=LibreChat-Cluster

For the ATLAS_MONGO_DB_URI, this should be transformed into:

mongodb+srv://user:<db_password>@librechat-cluster.e4wg71n.mongodb.net/DbName?retryWrites=true

Where DbName can be found by looking at the collections.