Vector Database Search Interrupted by Lengthy System Prompts

[chalitbkb]

What happened?

A bug was identified where setting a longer "system prompt" prevents vector data searches from being executed or skips the vector search component, while still processing the "system prompt" normally. However, when I tested with a smaller, not excessively long system prompt, vector data searches proceeded as usual.

Version Information

grep librechat

Steps to Reproduce

1. Create a System Prompt:

   • Set a system prompt that is intentionally long and detailed, exceeding the usual length you would use.

2. Initiate Vector Data Search:

   • Attempt to perform a vector data search using the AI system with the long system prompt in place.

3. Observe the Behavior:

   • Verify if the vector search component is skipped or not executed, while the system prompt still appears to be processed normally.

4. Modify the System Prompt:

   • Replace the long system prompt with a shorter and concise version that does not exceed typical length.

5. Retry Vector Data Search:

   • Perform the same vector data search with the new shorter system prompt.

6. Compare Results:

   • Note whether the vector data search executes successfully this time with the shorter system prompt.

What browsers are you seeing the problem on?

No response

Relevant log output

Screenshots

- When a large system prompt is set, the vector search is skipped

- Reducing the length of the system prompt allows vector search to function normally

Therefore, configuring system prompts to be longer based on user requirements should not impact vector searches.

Code of Conduct

• I agree to follow this project's Code of Conduct

[chalitbkb]

@danny-avila Could you please provide the reason for closing this thread? I hope you closed it by mistake and understand my explanation.

[danny-avila]

It's not closed, it was moved to a troubleshooting discussion. I don't believe this is a bug but rather a non-optimal way of going about using file search + agents.

When files are attached to an agent, part of the instructions is dedicated to saying which files are available. When you have very long instructions, and if none or even a small amount of it refers to file search usage, the longer prompt will overshadow the very basic dynamic prompt detailing available files and their usage with file search, which notably degrades performance for you. In this case, you may be explaining the files attached, but the LLM finds it has enough information to answer you.