404 error from RAG endpoint with custom embeddings

[erickrf]

What happened?

I have a Kubernetes deployment of LibreChat with self-hosted LLMs, and am trying to add the RAG endpoint. Whenever I try to add a file to a prompt, the frontend crashes, and the RAG API logs the following error:

Request POST http://rag-api.cluster.local:8000//embed - 404
//embed - {'id': '66e40907775cb306f9544665', 'username': 'erick', 'provider': 'local', 'email': 'my_email', 'iat': 1731343695, 'exp': 1731344595}

Steps to Reproduce

In my .env, I have the following RAG-related values:

RAG_API_URL=http://rag-api.cluster.local:8000
RAG_OPENAI_BASEURL=http://litellm.cluster.local:4000
EMBEDDINGS_PROVIDER=openai
EMBEDDINGS_MODEL=gte-large

What browsers are you seeing the problem on?

Firefox

Relevant log output

No response

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[alvieirajr]

@erickrf Are you using a custom RAG app ? The /embed endpoint asks for file_id, file, user, etc...
I don't think so the embedding interfer in some way on the librechat frontend ou backend. I build and integrated a custom RAG app with Librechat without worrying with kind of embedding or database my RAG app are using.

Can you give more details about you configuration, request and user operation on screen ?

[erickrf]

No, I'm using the librechat-rag, version 0.3.0 to be precise.

What details exactly do you need?

[erickrf]

I set the RAG address to a fake endpoint just running netcat to capture the exact request. Here is what I got:

POST /embed HTTP/1.1
Accept: application/json
Content-Type: multipart/form-data; boundary=--------------------------162312319343141653110784
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjY2ZTQwOTA3Nzc1Y2IzMDZmOTU0NDY2NSIsInVzZXJuYW1lIjoiZXJpY2siLCJwcm92aWRlciI6ImxvY2FsIiwiZW1haWwiOiJlcmljay5mb25zZWNhQHJlYWwtZGlnaXRhbC5kZSIsImlhdCI6MTczMTQwNDcwMSwiZXhwIjoxNzMxNDA1NjAxfQ.HuK4Yy0EDTsXihel3y1pWeyV57kD2jqs_M_4TD3NnPM
User-Agent: axios/1.7.4
Content-Length: 426
Accept-Encoding: gzip, compress, deflate, br
Host: rag-api.cluster.local:8001
Connection: keep-alive

----------------------------162312319343141653110784
Content-Disposition: form-data; name="file_id"

17f93530-5037-4023-b419-d59b6d89dd3c
----------------------------162312319343141653110784
Content-Disposition: form-data; name="file"; filename="test-rag.txt"
Content-Type: text/plain

The favorite number of Ashgil is 32.
The favorite number of Xiloch is 71.

----------------------------162312319343141653110784--

It seems to have all the required data, it would be weird if it didn't, as I didn't change anything regarding the RAG requests.

[alvieirajr]

@erickrf The 404 error means that doesn't exist the /embed router on http://rag-api.cluster.local:8000 for POST method.

[alvieirajr]

You request looks ok, but remember that you got 404 erro. Thia means that /embed router not exists on your RAG app. Em ter., 12 de nov. de 2024 06:57, Erick Fonseca ***@***.***> escreveu:

…

I set the RAG address to a fake endpoint just running netcat to capture the exact request. Here is what I got: POST /embed HTTP/1.1 Accept: application/json Content-Type: multipart/form-data; boundary=--------------------------162312319343141653110784 Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjY2ZTQwOTA3Nzc1Y2IzMDZmOTU0NDY2NSIsInVzZXJuYW1lIjoiZXJpY2siLCJwcm92aWRlciI6ImxvY2FsIiwiZW1haWwiOiJlcmljay5mb25zZWNhQHJlYWwtZGlnaXRhbC5kZSIsImlhdCI6MTczMTQwNDcwMSwiZXhwIjoxNzMxNDA1NjAxfQ.HuK4Yy0EDTsXihel3y1pWeyV57kD2jqs_M_4TD3NnPM User-Agent: axios/1.7.4 Content-Length: 426 Accept-Encoding: gzip, compress, deflate, br Host: rag-api.cluster.local:8001 Connection: keep-alive ----------------------------162312319343141653110784 Content-Disposition: form-data; name="file_id" 17f93530-5037-4023-b419-d59b6d89dd3c ----------------------------162312319343141653110784 Content-Disposition: form-data; name="file"; filename="test-rag.txt" Content-Type: text/plain The favorite number of Ashgil is 32. The favorite number of Xiloch is 71. ----------------------------162312319343141653110784-- It seems to have all the required data, it would be weird if it didn't, as I didn't change anything regarding the RAG requests. — Reply to this email directly, view it on GitHub <#4694 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABIYDRIM3277GGMKI4KONR32AHGI3AVCNFSM6AAAAABRSNQDZGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMRSGM4DKMY> . You are receiving this because you commented.Message ID: ***@***.***>

[erickrf]

Ok, not sure what happened but I'm not getting the 404 anymore after reinstalling my k8s deployment.

[alvieirajr]

Nice, problem resolved, right ? Just a curiosity, when are you setting the bearer token to your rag app ? In my case i needed to change the BaseClient.js. I create an axios interceptor that read the RAG_API_BEARER_TOKEN (this not exist in original librechat) env key and put the value on each request to RAG_API_URL. Something like this: // Intercepta o fetch

global.fetch = async (url, options = {}) => { // Adiciona o cabeçalho Authorization se o token estiver disponível if (process.env.RAG_API_URL && process.env.RAG_API_BEARER_TOKEN) { if (url.includes(process.env.RAG_API_URL)) { if (!options.headers) { options.headers = {}; } console.log('[./app/clients/BaseClient.js] - {Frontend using Aurora - Request To:', url); options.headers['Authorization'] = `Bearer ${process.env. RAG_API_BEARER_TOKEN}`; } }

Em ter., 12 de nov. de 2024 às 08:15, Erick Fonseca < ***@***.***> escreveu:

…

Ok, not sure what happened but I'm not getting the 404 anymore after reinstalling my k8s deployment. — Reply to this email directly, view it on GitHub <#4694 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABIYDRME6DT5PMJHV6IDP7T2AHPM5AVCNFSM6AAAAABRSNQDZGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMRSGQ3DMNI> . You are receiving this because you commented.Message ID: ***@***.***>

[erickrf]

I set the token in the RAG_OPENAI_API_KEY environment variable, which is set by Helm reading from Vault.

Not the most straightforward approach if you don't already have the infra set up for such use cases.