Agents and full text attachment (aka Mistral OCR) -> OCR context only transmitted to Agents on next chat messages

[amengus87]

What happened?

I discovered a bug with Agents and full text attachment (Mistral OCR) : it seems that OCR context is only transmitted to Agents on next chat messages.

Example :
"Give me all titles of this document" => "[Ran file_search] It seems that the document isn't currently available for me to search. Could you please upload the document so that I can extract the titles for you?"

[Ran file_search] should not occur as I'm using "full OCR context" and not the pgvector option to search in RAG

Version Information

cf03731cc8c87e2130a504b08afb2a7109684f52

Steps to Reproduce

1. Choose a PDF file to upload inside LibreChat (Agents)
2. Choose OCR option (not pgvector one)
3. Ask a question about the file uploaded : User : "Give me all titles of this document"
4. Agent : "[Ran file_search] It seems that the document isn't currently available for me to search. Could you please upload the document so that I can extract the titles for you?"
5. User : "Please give me them !"
6. Agent : "Here are the titles from the provided document: [...]" (Response cut for brevity)

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Nope, the OCR text is always included, the LLM gets confused when also adding the File Search capability. This can be prevented with better instructions.

[amengus87]

Have you some recommendations about better instructions to avoid confusion between File Search and OCR when both are enabled on agents?

Right now no special instructions are made about OCR and File Search to the agent, so I'm using default instructions hardcoded inside LibreChat Agents code.