Supporting RAG with Elasticsearch / OpenSearch

[sweitzel]

Given that I have a existing OpenSearch cluster with all my knowledge base data in it and with proper embeddings.
So basically I want that LibreChat accesses the data from OpenSearch (e.g. using Hybrid Search).

From what I saw at https://github.com/danny-avila/rag_api, the current RAG implementation is focusing on user provided documents, that will be used in the chat instead of relying on a centrally managed & externally provided vector database for RAG use case.

So my questions are:

1. Can I implement this myself already today? How hard would it be?
2. Is this scenario already supported by the RAG feature mentioned on the road map?
3. Or alternatively, could I use rag-api already today to re-index my knowledge base data into the pgvector store, and then use that data for generic RAG use case where LibreChat performs a query to extract the relevant knowledge base articles from the pgvector store and then provides this as context to the LLM?

[danny-avila]

From what I saw at https://github.com/danny-avila/rag_api, the current RAG implementation is focusing on user provided documents, that will be used in the chat instead of relying on a centrally managed & externally provided vector database for RAG use case.

It's possible to still provide "knowledge bases" or "data sets" for a collection of vectors that is centrally managed and I plan to provide that with the current implementation.

So my questions are:

1. Can I implement this myself already today? How hard would it be?

Depends on your level of expertise with python. I can imagine it working well and not too difficult but will be involved in at least testing the whole setupL https://python.langchain.com/docs/integrations/vectorstores/elasticsearch

Fortunately, elasticsearch looks to be compatible with the general approach I use which is similar to LangChain's implementation of the Indexing API: https://python.langchain.com/docs/modules/data_connection/indexing#requirements

2. Is this scenario already supported by the RAG feature mentioned on the road map?

No, currently only PGVector is supported wholesale.

3. Or alternatively, could I use rag-api already today to re-index my knowledge base data into the pgvector store, and then use that data for generic RAG use case where LibreChat performs a query to extract the relevant knowledge base articles from the pgvector store and then provides this as context to the LLM?

This would be somewhat involved but not difficult as you would need to sync up with the mongo database as well for the intended, implemented design.

Alternative to all this, you can just inject RAG at any point of the completion process, before the actual API call to the AI service, hacking in your own API, especially if you are more concerned about "general" knowledge and not selecting files.

Plugin is also an option.

[sweitzel]

Thank you for the detailed reply!

The alternatives you mentioned (injection or plugin) actually seem like the low hanging fruit for now. I will check both options out and share the results.

Of course if rag_api would support the external Elasticsearch integration in the long run, that avoids the need for these customizations and may provide a more efficient RAG approach.

[danny-avila]

Thank you for the detailed reply!

The alternatives you mentioned (injection or plugin) actually seem like the low hanging fruit for now. I will check both options out and share the results.

Thank you for the discussion. I may also look into meilisearch integration since we already include it in the project: https://python.langchain.com/docs/integrations/vectorstores/meilisearch

[tatwong]

Is that means I can use the same MelliSearch configuration to configure OpenSearch?