custom prompt template with a retriever pipeline and agent with memory

[HGamalElDin]

Hello guys!

Please confirm if I can not implement the RAG pipeline with a chat memory (without using agents)?
Notes that might give you intuition on my case:

1. I have a custom prompt template as follows:

[INS]<<SYS>>

You are the *** assistant bot. You are representing a bot answering ***'s employees' frequently asked questions from the context of ***'s handbooks.
Derive the concise and exact answer for the human's question concisely from the given context. You must answer the question exactly without including any redundant additional information.
Rely exclusively on the provided context. Don't include any additional text out of the question's scope.
Maintain an impartial and journalistic tone. Avoid repetition or redundant information. Do not mention anything regarding the context or *** handbooks unless it's requested by the human.
If you can not find the answer within the provided context, in this case just say that ```**** is still in learning mode. You can switch to ChatGPT mode for general knowledge and public information.```
If a summary of the context is requested by the human only, summarize the context without omitting important information in a structured answer.
Never include any emojis in your responses.
Your answer must be structured as better as it can look. 
If the Human greeted you, greet him back in a journalistic way.
<</SYS>>


Context: {documents};
Human Question: {input};[/INS]
According to the provided context, Your Answer:

So here I have two variables.
Question 1: how to provide the documents variable to the agent you created in your comment?

I tried the following snippet, however, the default memory injected in the agent doesn't store the Human input at all.

from haystack.nodes import PromptNode  
  
model_name = 'gpt-3.5-turbo'  
prompt_node = PromptNode(model_name, api_key="my_key", default_prompt_template=r"QA-Prompt.yaml", max_length=256)

from haystack.agents.conversational import ConversationalAgent  
conversational_agent = ConversationalAgent(prompt_node=prompt_node, memory=summary_memory)

conversational_agent.run({"input": "what is france's captial?", "documents":"""Working Days: Sunday to Thursday
Working Hours: 9:00 AM to 5:00 PM

Working Days (Ramadan): Sunday to Thursday
Working Hours (Ramadan): 10:30 AM to 4:00 PM
"""})

When I call
conversational_agent.memory.load()

I got """Human: input\nAI: Paris is the capital of France.\nHuman: input\nAI: During Ramadan, working hours are 10:30 AM to 4:00 PM, Sunday to Thursday.\nHuman: input\nAI: France's capital is Paris.\n"""

Also, the prompt template I provided should control the responses that no random answers should be generated! only from the context or the response should be a specific message as provided! it works well if I only use the prompt node, but with agent It's like the prompt template doesn't work, what could be the possible cause of this?

[vblagoje]

Hey @HGamalElDin, you basically want to have a conversational chat agent that uses RAG pipeline, right? If so, then I think this is possible if you use an RAG pipeline as a conversation agent tool. Let me check if there are some tutorials with exactly this use case.

[HGamalElDin]

Hey @vblagoje, the problem with this approach is that the agent takes an object of the LLM without my custom prompt template, my use case is supposed that the bot shouldn't respond any questions that aren't in the provided context (got from the RAG pipeline), but the general LLM passed to the agent makes it respond to any question like chatgpt, if you understand me.

So I tried this approach, however I see it doesn't follow exactly my custom prompt template specified in the prompt node which is placed in the last layer of the RAG.

[vblagoje]

Yes, but you can set the default prompt in PromptNode via default_prompt_template init parameter. The best option would be to share some relevant snippets of code so we can look at it together....