Local RAG does not work with custom endpoints

[aleibovici]

What happened?

I am currently using a local RAG for creating embeddings, and this process is successfully completing. However, when I attempt to ask a question about a document, the custom endpoint LLM (irrespective of the model) responds by indicating that the document was not supplied and this is the first interaction. In the same message context, when I switch to OpenAI or Anthropic, the document context is correctly sent to the LLM.

Steps to Reproduce

1. Configure LibreChat for local RAG with "nomic-embed-text"
   here is .env config:
   RAG_API_URL=http://host.docker.internal:8000
   EMBEDDINGS_PROVIDER=ollama
   OLLAMA_BASE_URL=http://10.69.240.50:11434
   EMBEDDINGS_MODEL=nomic-embed-text
   RAG_USE_FULL_CONTEXT=true

2. Configure local LLM (any model)

3. Test with PDF

What browsers are you seeing the problem on?

No response

Relevant log output

rag_api           | 2024-10-21 18:50:07,412 - httpx - INFO - HTTP Request: POST http://10.69.240.50:11434/api/embed "HTTP/1.1 200 OK"
rag_api           | 2024-10-21 18:50:07,423 - root - INFO - Request POST http://rag_api:8000/embed - 200
LibreChat         | 2024-10-21T18:50:07.425Z debug: File processed successfully.
LibreChat         | {
LibreChat         |   status: true,
LibreChat         |   file_id: "84518d58-e9c5-4527-a6ad-0e08da94afdf",
LibreChat         |   filename: "pdf-sample_0.pdf",
LibreChat         |   known_type: true,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.059Z debug: [AskController]
LibreChat         | {
LibreChat         |   text: "What is this file about?",
LibreChat         |   conversationId: null,
LibreChat         |   endpoint: "SecureLlama",
LibreChat         |   endpointType: "custom",
LibreChat         |   resendFiles: true,
LibreChat         |     modelOptions.model: "qwen2.5:14b",
LibreChat         |   artifactsPrompt: "The assistant can create and reference artifacts during conversations.\n\nArtifacts are for substantia... [truncated]",
LibreChat         |   modelsConfig: "exists",
LibreChat         |   attachments: [object Promise],
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.061Z debug: [BaseClient] Loading history:
LibreChat         | {
LibreChat         |   conversationId: "8e9c5239-d614-4c95-856f-74e8aac9cc51",
LibreChat         |   parentMessageId: "00000000-0000-0000-0000-000000000000",
LibreChat         | }
rag_api           | 2024-10-21 18:50:17,076 - root - INFO - Request GET http://rag_api:8000/documents/84518d58-e9c5-4527-a6ad-0e08da94afdf/context - 200
LibreChat         | 2024-10-21T18:50:17.093Z debug: [BaseClient] instructions tokenCount: 5078
LibreChat         | 2024-10-21T18:50:17.093Z debug: [BaseClient] Context Count (1/2)
LibreChat         | {
LibreChat         |   remainingContextTokens: 4082,
LibreChat         |   maxContextTokens: 4095,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.094Z debug: [BaseClient] Difference between original payload (2) and context (1): 1
LibreChat         | 2024-10-21T18:50:17.094Z debug: [BaseClient] Context Count (2/2)
LibreChat         | {
LibreChat         |   remainingContextTokens: 4082,
LibreChat         |   maxContextTokens: 4095,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.095Z debug: [BaseClient] tokenCountMap:
LibreChat         | {
LibreChat         |   43b570b7-e08a-47e2-b717-29bfc657361a: 10,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.095Z debug: [BaseClient]
LibreChat         | {
LibreChat         |   promptTokens: 13,
LibreChat         |   remainingContextTokens: 4082,
LibreChat         |   payloadSize: 1,
LibreChat         |   maxContextTokens: 4095,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.096Z debug: [BaseClient] tokenCountMap
LibreChat         | {
LibreChat         |   43b570b7-e08a-47e2-b717-29bfc657361a: 10,
LibreChat         |   instructions: 5078,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.096Z debug: [BaseClient] userMessage
LibreChat         | {
LibreChat         |   messageId: "43b570b7-e08a-47e2-b717-29bfc657361a",
LibreChat         |   parentMessageId: "00000000-0000-0000-0000-000000000000",
LibreChat         |   conversationId: "8e9c5239-d614-4c95-856f-74e8aac9cc51",
LibreChat         |   sender: "User",
LibreChat         |   text: "What is this file about?",
LibreChat         |   isCreatedByUser: true,
LibreChat         |   image_urls: undefined,
LibreChat         |   tokenCount: 10,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.097Z debug: [OpenAIClient] chatCompletion
LibreChat         | {
LibreChat         |   baseURL: "http://10.69.240.50:11434/v1",
LibreChat         |     modelOptions.model: "qwen2.5:14b",
LibreChat         |     modelOptions.user: "66f6376dba28a6a7dbd623a1",
LibreChat         |     modelOptions.stream: true,
LibreChat         |     // 1 message(s)
LibreChat         |     modelOptions.messages: [{"role":"user","content":"What is this file about?"}],
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.100Z debug: Making request to http://10.69.240.50:11434/v1/chat/completions
LibreChat         | 2024-10-21T18:50:17.106Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
chat-meilisearch  | 2024-10-21T18:50:17.112989Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=555µs time.idle=3.49ms
chat-meilisearch  | 2024-10-21T18:50:17.117861Z  WARN HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/8e9c5239-d614-4c95-856f-74e8aac9cc51 query_parameters= user_agent=node status_code=404 error=Document `8e9c5239-d614-4c95-856f-74e8aac9cc51` not found.}: tracing_actix_web::middleware: Error encountered while processing the incoming HTTP request: ResponseError { code: 404, message: "Document `8e9c5239-d614-4c95-856f-74e8aac9cc51` not found.", error_code: "document_not_found", error_type: "invalid_request", error_link: "https://docs.meilisearch.com/errors#document_not_found" }
chat-meilisearch  | 2024-10-21T18:50:17.118197Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/8e9c5239-d614-4c95-856f-74e8aac9cc51 query_parameters= user_agent=node status_code=404 error=Document `8e9c5239-d614-4c95-856f-74e8aac9cc51` not found.}: meilisearch: close time.busy=441µs time.idle=1.17ms
chat-meilisearch  | 2024-10-21T18:50:17.126237Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/convos/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=338µs time.idle=3.08ms
chat-meilisearch  | 2024-10-21T18:50:17.146372Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 298 }
chat-meilisearch  | 2024-10-21T18:50:17.160841Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
chat-meilisearch  | 2024-10-21T18:50:17.175675Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 48 }
chat-meilisearch  | 2024-10-21T18:50:17.182422Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
LibreChat         | 2024-10-21T18:50:17.760Z debug: [OpenAIClient] chatCompletion response
LibreChat         | {
LibreChat         |   object: "chat.completion",
LibreChat         |   id: "chatcmpl-956",
LibreChat         |   // 1 choice(s)
LibreChat         |   choices: [{"message":{"role":"assistant","content":"I'm here to help, but I need more information for me to as... [truncated]],
LibreChat         |   created: 1729536617,
LibreChat         |   model: "qwen2.5:14b",
LibreChat         |   system_fingerprint: "fp_ollama",
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.761Z debug: [spendTokens] conversationId: 8e9c5239-d614-4c95-856f-74e8aac9cc51 | Context: message | Token usage:
LibreChat         | {
LibreChat         |   promptTokens: 13,
LibreChat         |   completionTokens: 35,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.770Z debug: [spendTokens] No transactions incurred against balance
LibreChat         | 2024-10-21T18:50:17.775Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
chat-meilisearch  | 2024-10-21T18:50:17.781832Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/8e9c5239-d614-4c95-856f-74e8aac9cc51 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=131µs time.idle=370µs
chat-meilisearch  | 2024-10-21T18:50:17.782120Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=1.17ms time.idle=5.98ms
LibreChat         | 2024-10-21T18:50:17.785Z debug: [AskController] Request closed
chat-meilisearch  | 2024-10-21T18:50:17.795237Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=285µs time.idle=4.10ms
LibreChat         | 2024-10-21T18:50:17.799Z debug: [OpenAIClient] chatCompletion
LibreChat         | {
LibreChat         |   baseURL: "http://10.69.240.50:11434/v1",
LibreChat         |     modelOptions.model: "qwen2.5:14b",
LibreChat         |     modelOptions.user: "66f6376dba28a6a7dbd623a1",
LibreChat         |     modelOptions.temperature: 0.2,
LibreChat         |     modelOptions.presence_penalty: 0,
LibreChat         |     modelOptions.frequency_penalty: 0,
LibreChat         |     modelOptions.max_tokens: 16,
LibreChat         |     // 1 message(s)
LibreChat         |     modelOptions.messages: [{"role":"system","content":"Please generate a concise, 5-word-or-less title for the conversation, us... [truncated]],
LibreChat         | }
chat-meilisearch  | 2024-10-21T18:50:17.800688Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=362µs time.idle=3.40ms
LibreChat         | 2024-10-21T18:50:17.800Z debug: Making request to http://10.69.240.50:11434/v1/chat/completions
chat-meilisearch  | 2024-10-21T18:50:17.816433Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 299 }
chat-meilisearch  | 2024-10-21T18:50:17.831070Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
chat-meilisearch  | 2024-10-21T18:50:17.865165Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 2, number_of_documents: 299 }
chat-meilisearch  | 2024-10-21T18:50:17.870536Z  INFO index_scheduler: A batch of tasks was successfully completed with 2 successful tasks and 0 failed tasks.
LibreChat         | 2024-10-21T18:50:17.966Z debug: [OpenAIClient] chatCompletion response
LibreChat         | {
LibreChat         |   id: "chatcmpl-880",
LibreChat         |   object: "chat.completion",
LibreChat         |   created: 1729536617,
LibreChat         |   model: "qwen2.5:14b",
LibreChat         |   system_fingerprint: "fp_ollama",
LibreChat         |   // 1 choice(s)
LibreChat         |   choices: [{"index":0,"message":{"role":"assistant","content":"File Inquiry Assistance Needed"},"finish_reason"... [truncated]],
LibreChat         |     usage.prompt_tokens: 108,
LibreChat         |     usage.completion_tokens: 5,
LibreChat         |     usage.total_tokens: 113,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.967Z debug: [spendTokens] conversationId: 8e9c5239-d614-4c95-856f-74e8aac9cc51 | Context: title | Token usage:
LibreChat         | {
LibreChat         |   promptTokens: 104,
LibreChat         |   completionTokens: 4,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:17.969Z debug: [OpenAIClient] Convo Title: File Inquiry Assistance Needed
LibreChat         | 2024-10-21T18:50:17.969Z debug: [saveConvo] api/server/services/Endpoints/openAI/addTitle.js
LibreChat         | 2024-10-21T18:50:17.976Z debug: [spendTokens] No transactions incurred against balance
chat-meilisearch  | 2024-10-21T18:50:17.981135Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/8e9c5239-d614-4c95-856f-74e8aac9cc51 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=133µs time.idle=208µs
chat-meilisearch  | 2024-10-21T18:50:17.987429Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/convos/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=269µs time.idle=2.79ms
chat-meilisearch  | 2024-10-21T18:50:18.001197Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 48 }
chat-meilisearch  | 2024-10-21T18:50:18.007129Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
LibreChat         | 2024-10-21T18:50:33.175Z debug: [AskController]
LibreChat         | {
LibreChat         |   text: "What is this file about?",
LibreChat         |   conversationId: "8e9c5239-d614-4c95-856f-74e8aac9cc51",
LibreChat         |   endpoint: "openAI",
LibreChat         |   resendFiles: true,
LibreChat         |     modelOptions.model: "gpt-4o",
LibreChat         |   artifactsPrompt: "The assistant can create and reference artifacts during conversations.\n\nArtifacts are for substantia... [truncated]",
LibreChat         |   modelsConfig: "exists",
LibreChat         | }
LibreChat         | 2024-10-21T18:50:33.177Z debug: [BaseClient] Loading history:
LibreChat         | {
LibreChat         |   conversationId: "8e9c5239-d614-4c95-856f-74e8aac9cc51",
LibreChat         |   parentMessageId: "5de203c1-bf8e-4668-9110-9cf9478ecff5",
LibreChat         | }
rag_api           | 2024-10-21 18:50:33,449 - root - INFO - Request GET http://rag_api:8000/documents/84518d58-e9c5-4527-a6ad-0e08da94afdf/context - 200
LibreChat         | 2024-10-21T18:50:33.466Z debug: [BaseClient] instructions tokenCount: 5143
LibreChat         | 2024-10-21T18:50:33.467Z debug: [BaseClient] Context Count (1/2)
LibreChat         | {
LibreChat         |   remainingContextTokens: 122295,
LibreChat         |   maxContextTokens: 127500,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:33.467Z debug: [BaseClient] Context Count (2/2)
LibreChat         | {
LibreChat         |   remainingContextTokens: 122295,
LibreChat         |   maxContextTokens: 127500,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:33.467Z debug: [BaseClient] tokenCountMap:
LibreChat         | {
LibreChat         |   43b570b7-e08a-47e2-b717-29bfc657361a: 10,
LibreChat         |   5de203c1-bf8e-4668-9110-9cf9478ecff5: 39,
LibreChat         |   e9403db8-8d38-4d92-be4f-643b2a55a8f2: 10,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:33.467Z debug: [BaseClient]
LibreChat         | {
LibreChat         |   promptTokens: 5205,
LibreChat         |   remainingContextTokens: 122295,
LibreChat         |   payloadSize: 4,
LibreChat         |   maxContextTokens: 127500,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:33.468Z debug: [BaseClient] tokenCountMap
LibreChat         | {
LibreChat         |   43b570b7-e08a-47e2-b717-29bfc657361a: 10,
LibreChat         |   5de203c1-bf8e-4668-9110-9cf9478ecff5: 39,
LibreChat         |   e9403db8-8d38-4d92-be4f-643b2a55a8f2: 10,
LibreChat         |   instructions: 5143,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:33.468Z debug: [BaseClient] userMessage
LibreChat         | {
LibreChat         |   messageId: "e9403db8-8d38-4d92-be4f-643b2a55a8f2",
LibreChat         |   parentMessageId: "5de203c1-bf8e-4668-9110-9cf9478ecff5",
LibreChat         |   conversationId: "8e9c5239-d614-4c95-856f-74e8aac9cc51",
LibreChat         |   sender: "User",
LibreChat         |   text: "What is this file about?",
LibreChat         |   isCreatedByUser: true,
LibreChat         |   tokenCount: 10,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:33.468Z debug: [BaseClient] Skipping 43b570b7-e08a-47e2-b717-29bfc657361a: already had a token count.
LibreChat         | 2024-10-21T18:50:33.468Z debug: [BaseClient] Skipping 5de203c1-bf8e-4668-9110-9cf9478ecff5: already had a token count.
LibreChat         | 2024-10-21T18:50:33.470Z debug: [OpenAIClient] chatCompletion
LibreChat         | {
LibreChat         |   baseURL: "https://api.openai.com/v1",
LibreChat         |     modelOptions.model: "gpt-4o",
LibreChat         |     modelOptions.user: "66f6376dba28a6a7dbd623a1",
LibreChat         |     modelOptions.stream: true,
LibreChat         |     // 4 message(s)
LibreChat         |     modelOptions.messages: [{"role":"user","content":"What is this file about?"},{"role":"assistant","content":"I'm here to help, but I need more information for me to assist you ef... [truncated],{"role":"system","name":"instructions","content":"Instructions:\nThe user has attached a file to the... [truncated],{"role":"user","content":"What is this file about?"}],
LibreChat         | }
LibreChat         | 2024-10-21T18:50:33.476Z debug: Making request to https://api.openai.com/v1/chat/completions
LibreChat         | 2024-10-21T18:50:33.482Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
chat-meilisearch  | 2024-10-21T18:50:33.491954Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/8e9c5239-d614-4c95-856f-74e8aac9cc51 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=259µs time.idle=686µs
chat-meilisearch  | 2024-10-21T18:50:33.494474Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=633µs time.idle=3.64ms
chat-meilisearch  | 2024-10-21T18:50:33.526940Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 300 }
chat-meilisearch  | 2024-10-21T18:50:33.538831Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
LibreChat         | 2024-10-21T18:50:35.167Z debug: [OpenAIClient] chatCompletion response
LibreChat         | {
LibreChat         |   object: "chat.completion",
LibreChat         |   id: "chatcmpl-AKrm551KjulNO5cduNBU3OAC1loDP",
LibreChat         |   // 1 choice(s)
LibreChat         |   choices: [{"message":{"role":"assistant","content":"The file named \"pdf-sample_0.pdf\" appears to be a dummy ... [truncated]],
LibreChat         |   created: 1729536633,
LibreChat         |   model: "gpt-4o-2024-08-06",
LibreChat         |   system_fingerprint: "fp_a7d06e42a7",
LibreChat         | }
LibreChat         | 2024-10-21T18:50:35.480Z debug: [spendTokens] conversationId: 8e9c5239-d614-4c95-856f-74e8aac9cc51 | Context: message | Token usage:
LibreChat         | {
LibreChat         |   promptTokens: 5205,
LibreChat         |   completionTokens: 40,
LibreChat         | }
LibreChat         | 2024-10-21T18:50:35.488Z debug: [spendTokens] No transactions incurred against balance
LibreChat         | 2024-10-21T18:50:35.494Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
chat-meilisearch  | 2024-10-21T18:50:35.499973Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=318µs time.idle=5.51ms
chat-meilisearch  | 2024-10-21T18:50:35.501730Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/8e9c5239-d614-4c95-856f-74e8aac9cc51 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=102µs time.idle=328µs
LibreChat         | 2024-10-21T18:50:35.506Z debug: [AskController] Request closed
chat-meilisearch  | 2024-10-21T18:50:35.515921Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=367µs time.idle=2.89ms
chat-meilisearch  | 2024-10-21T18:50:35.521775Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=279µs time.idle=3.65ms
chat-meilisearch  | 2024-10-21T18:50:35.534778Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 301 }
chat-meilisearch  | 2024-10-21T18:50:35.553272Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
chat-meilisearch  | 2024-10-21T18:50:35.587148Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 2, number_of_documents: 301 }
chat-meilisearch  | 2024-10-21T18:50:35.600795Z  INFO index_scheduler: A batch of tasks was successfully completed with 2 successful tasks and 0 failed tasks.
vectordb          | 2024-10-21 18:50:53.376 UTC [27] LOG:  checkpoint starting: time
chat-mongodb      | {"t":{"$date":"2024-10-21T18:50:54.410+00:00"},"s":"I",  "c":"WTCHKPT",  "id":22430,   "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":{"ts_sec":1729536654,"ts_usec":410845,"thread":"1:0x7f67df5466c0","session_name":"WT_SESSION.checkpoint","category":"WT_VERB_CHECKPOINT_PROGRESS","category_id":7,"verbose_level":"DEBUG_1","verbose_level_id":1,"msg":"saving checkpoint snapshot min: 68, snapshot max: 68 snapshot count: 0, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0) base write gen: 127043"}}}
vectordb          | 2024-10-21 18:50:54.596 UTC [27] LOG:  checkpoint complete: wrote 15 buffers (0.1%); 0 WAL file(s) added, 0 removed, 0 recycled; write=1.207 s, sync=0.005 s, total=1.221 s; sync files=9, longest=0.004 s, average=0.001 s; distance=20 kB, estimate=20 kB
chat-mongodb      | {"t":{"$date":"2024-10-21T18:51:54.462+00:00"},"s":"I",  "c":"WTCHKPT",  "id":22430,   "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":{"ts_sec":1729536714,"ts_usec":462273,"thread":"1:0x7f67df5466c0","session_name":"WT_SESSION.checkpoint","category":"WT_VERB_CHECKPOINT_PROGRESS","category_id":7,"verbose_level":"DEBUG_1","verbose_level_id":1,"msg":"saving checkpoint snapshot min: 69, snapshot max: 69 snapshot count: 0, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0) base write gen: 127043"}}}

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

It really depends on well the model interprets the RAG prompt, as I don't see any errors in your logs:

[aleibovici]

Hi @danny-avila , your help would be much appreciated.

The same PDF document is being uploaded using ollama nomic embedding.

• First, my query is using Ollama to extract the content of the document. (I'm sorry, but you haven't provided a document for me to review...) and we get same result for any of the models.

Here is the error:

chat-meilisearch | 2024-11-13T01:15:22.532555Z WARN HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/2812482a-6731-4481-a7d2-1b86364090ab query_parameters= user_agent=node status_code=404 error=Document 2812482a-6731-4481-a7d2-1b86364090ab not found.}: tracing_actix_web::middleware: Error encountered while processing the incoming HTTP request: ResponseError { code: 404, message: "Document 2812482a-6731-4481-a7d2-1b86364090ab not found.", error_code: "document_not_found", error_type: "invalid_request", error_link: "https://docs.meilisearch.com/errors#document_not_found" }
chat-meilisearch | 2024-11-13T01:15:22.532821Z INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/2812482a-6731-4481-a7d2-1b86364090ab query_parameters= user_agent=node status_code=404 error=Document 2812482a-6731-4481-a7d2-1b86364090ab not found.}: meilisearch: close time.busy=233µs time.idle=268µs
chat-meilisearch | 2024-11-13T01:15:22.544505Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/convos/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=288µs time.idle=4.38ms

• The second query is using OpenAI for the same document and it works fine.

Here the config:

endpoints:
custom:
- name: "Secure LLM"
apiKey: "ollama"
baseURL: "http://10.69.240.50:11434/v1/"
models:
default: [
"gemma2:27b",
"qwen2.5:14b",
"llama3.2:3b",
]
fetch: false
titleConvo: true
titleModel: "current_model"
summarize: false
summaryModel: "current_model"
forcePrompt: false
modelDisplayLabel: "Secure LLM"

Here is the full logging:

librechat_error_log.txt