Customizing Prompts for MapReduce and Refine Chains

[RRaffay]

I'm trying to figure out how to pass custom instructions while using pre-built abstractions such as RetrievalQA. More specifically, I want the map-reduce or refine chain to analyze the documents keeping in mind some context. I know this should use the chain_type_kwargs along with PromptTemplate.from_template(template), but I'm unsure how to incorporate different custom prompts for different parts of the map reduce chain. For example, I want to change the

"Use the following portion of a long document to see if any of the text is relevant to answer the question.
Return any relevant text verbatim."

by appending some context about the documents. Then, I want to change

"Given the following extracted parts of a long document and a question, create a final answer.
If you don't know the answer, just say that you don't know. Don't try to make up an answer."

by appending different variation of context.

[dosubot[bot]]

🤖

Hello,

To customize different prompts for different parts of the map-reduce chain in the RetrievalQA abstraction of LangChain, you can create different instances of PromptTemplate with different templates. Here's an example of how you can append specific context to the existing instructions:

from langchain.prompts import PromptTemplate

# Original prompt
prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.

{context}

Question: {question}
Helpful Answer:"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

# Customized prompt for a specific part of the map-reduce chain
custom_prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.

{context}

Additional Context: {additional_context}

Question: {question}
Helpful Answer:"""
CUSTOM_PROMPT = PromptTemplate(
    template=custom_prompt_template, input_variables=["context", "additional_context", "question"]
)

In this example, CUSTOM_PROMPT is a new instance of PromptTemplate that includes an additional context in the template. You can use this customized prompt in the specific part of the map-reduce chain where you want to append the specific context to the existing instructions.

To pass custom prompts to the RetrievalQA abstraction in LangChain, you can use the from_llm class method of the BaseRetrievalQA class. This method takes an optional prompt parameter, which you can use to pass your custom PromptTemplate instance. If you don't provide a prompt, the method will use the default prompt for the given language model.

Here's an example of how you can create a custom prompt and pass it to the from_llm method:

from langchain.llms import OpenAI
from langchain.chains import RetrievalQA
from langchain.prompts import PromptTemplate

custom_prompt_template = """Here is some context:

{context}

Please answer the following question: {question}
"""
CUSTOM_PROMPT = PromptTemplate(
    template=custom_prompt_template, input_variables=["context", "question"]
)

retrievalQA = RetrievalQA.from_llm(llm=OpenAI(), prompt=CUSTOM_PROMPT)

In this example, CUSTOM_PROMPT is a new instance of PromptTemplate that uses a custom prompt template. The placeholders {context} and {question} in the template will be replaced with the actual context and question when the prompt is used. The from_llm method then creates a new instance of RetrievalQA using the given language model and custom prompt.

I hope this helps! Let me know if you have any other questions.

Sources

• Issue: Changing Prompt (from Default) when Using ConversationalRetrievalChain?
• libs/langchain/langchain/chains/retrieval_qa/prompt.py
• libs/langchain/langchain/prompts/prompt.py
• libs/langchain/langchain/chains/retrieval_qa/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[RRaffay]

@dosu-bot What if I want to use RetrievalQA.from_chain_type, keep the prompt used by the chain type, say map_reduce, and only append the context.

[dosubot[bot]]

🤖

Based on the context provided, there are a couple of ways you can append additional context to the prompt used by the chain type in the RetrievalQA.from_chain_type method while keeping the original prompt.

One way is to modify the PROMPT template in the prompt.py file. You can add your additional context in the prompt_template string. Here's how you can do it:

prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.

{context}

Additional Context: {additional_context}

Question: {question}
Helpful Answer:"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "additional_context", "question"]
)

In this code, {additional_context} is a placeholder for the additional context that you want to append. You also need to add "additional_context" to the input_variables list in the PromptTemplate constructor. This tells the PromptTemplate that it should expect an additional input variable named additional_context when it's used to generate a prompt.

Please note that you will need to provide this additional_context when you are using the PROMPT to generate a prompt.

In the context shared, the PromptTemplate class also provides a format method that you can use to format the prompt with the inputs. You can use this method to provide the additional_context when you are using the PROMPT to generate a prompt. Here's how you can do it:

PROMPT.format(context="Your original context", additional_context="Your additional context", question="Your question")

This will return a formatted string that includes your original context, the additional context, and the question.

Another way is to change the main prompt in ConversationalRetrievalChain by passing it in via combine_docs_chain_kwargs if you instantiate the chain using from_llm. Here is the code snippet:

ConversationalRetrievalChain.from_llm(combine_docs_chain_kwargs={"prompt": your_prompt}))

Alternatively, you can use load_qa_chain with memory and add your custom prompt. Here is an example:

from langchain.chains.question_answering import load_qa_chain
template = """
  {Your_Prompt}
  
  CONTEXT:
  {context}
  
  QUESTION: 
  {query}

  CHAT HISTORY: 
  {chat_history}
  
  ANSWER:
  """

prompt = PromptTemplate(input_variables=["chat_history", "query", "context"], template=template)
memory = ConversationBufferMemory(memory_key="chat_history", input_key="query")

chain = load_qa_chain(ChatOpenAI(temperature=0), chain_type="stuff", memory=memory, prompt=prompt)

I hope this helps! Let me know if you have any other questions.

Sources

• Issue: Changing Prompt (from Default) when Using ConversationalRetrievalChain?
• ValidationError: 1 validation error for ConversationalRetrievalChain chain_type_kwargs extra fields not permitted (type=value_error.extra)
• libs/langchain/langchain/chains/retrieval_qa/prompt.py
• libs/langchain/langchain/prompts/prompt.py
• libs/langchain/langchain/chains/retrieval_qa/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[RRaffay]

Probably not the cleanest way, but used the following workaround to add customization to the retrievalQA chain with map-reduce:

qa_chain_mr_test = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=retriever,
    chain_type="map_reduce"
    )

# existing_prompt_template = 
system_message = qa_chain_mr_test.combine_documents_chain.llm_chain.prompt.messages[0].prompt.template + custom_prompt 
qa_chain_mr_test.combine_documents_chain.llm_chain.prompt.messages[0].prompt.template = system_message