[Question]: How to add custom RAG?

[jayaprakash1207]

What is your question?

I have a separate service with a post API consisting of the endpoint - /query which takes the text from the input of the user and gives a LLM generated response. I want to integrate this as a custom endpoint. How do I do it?

More Details

Like I have tried to configure the custom endpoint in the librechat.yml and made some custom changes in the service controllers and app client but I'm facing serveral issues regarding the conversationID and facing few mishaps in the request and response cycle.

It would be great to know a solution for this real quick!

Thanks!

What is the main subject of your question?

Endpoints

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

I mean, if it works the same as the current RAG server, you would just need to change RAG_API_URL

I want to integrate this as a custom endpoint. How do I do it?

There's a similar discussion on this topic:

#2839

[keerthana-5170]

Hi @danny-avila , working on integrating a similar custom RAG API with Librechat, its working is not similar to current RAG server since it stores embeddings and answers for the pdfs uploaded, whereas our POST RAG API which answers from the pre ingested set of documents from a vector db(which doesn't require any documents to be uploaded from user end). Attaching a working of our RAG architecture.

Can you confirm , if such an custom post RAG API can be integrated as an endpoint with libre chat?

[acbsn]

this would be extremely useful since I also have a similar set up where my RAG system is already up and running so I just need the answer (and maybe it would be great to be able to display in the UI the Document excerpts used to generate the answer)