Improve RAG mechanism

[PSchmiedmayer]

Problem

LLM sometimes doesn't fully pick up the context in the RAG.

Solution

As discussed in the meeting:

• Improve main system prompt
• Improve ways to inject RAG in conversation, additional system prompts
• Use more context than user message, summarize conversation and/or use LLM to identify query
• Improve chunking; move to Python function

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct and Contributing Guidelines