RetrievalQA while using multiple input parameters in Promptemplate

[shikharpant]

Hi, I am having trouble with using multiple input variables with RetrievalQA chain.
My code is as below

from langchain.prompts import PromptTemplate
def get_prompt_template(model):
    if model == 'gpt-3.5-turbo':
        instruction = """
        You are a helpful, respectful assistant.
        Answer given question based on CONTEXT and openai_response.

        If you cant frame an answer just say I dont know

        Question: {query}
        Context: {context}
        openai_response: {openai_response}
        Response: """
        
        prompt_template = PromptTemplate(template=instruction, 
                                         input_variables=["context", "query"],
                                         partial_variables={"openai_response":openai_response}
                                         )
        prompt = {"prompt": prompt_template}

        return prompt
        
from langchain.llms import OpenAI
def get_direct_openai_answer(question):
  chat_openai = OpenAI(temperature=0.01)
  response_openai = chat_openai(question)
  formatted_response = {
      'page_content': response_openai,
      'metadata': {'source': 'OpenAI API Call'}
  }
  return formatted_response

from langchain.chat_models import ChatOpenAI
from langchain.chains import RetrievalQA
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

query ='query'
model = 'gpt-3.5-turbo'    
local_llm = ChatOpenAI(model="gpt-3.5-turbo-16k", temperature=0.6)
openai_response=get_direct_openai_answer(query)
chain_type_kwargs = get_prompt_template(model)
qa_chain = RetrievalQA.from_chain_type(llm=local_llm,
                                      chain_type="stuff",
                                      retriever=retriever,
                                      return_source_documents=True,
                                      chain_type_kwargs=chain_type_kwargs,
                                      )
                                      
print((qa_chain({"query": query})))

The idea was that along with the output of vector stores, I wanted the LLM to take standard API response also into account. And maybe later modify it to include Websearch response with different weights to such responses.

[dosubot[bot]]

🤖

Hello,

From your code, it seems like you're trying to use the openai_response as a partial variable in your PromptTemplate. However, openai_response is not defined in the scope where you're creating the PromptTemplate.

To use multiple input variables with the RetrievalQA chain in LangChain, you need to modify the input_variables parameter in the PromptTemplate object. This parameter is a list that specifies the names of the variables that will be used in the prompt template.

In your case, if you want to include both the output of vector stores and the standard API response, you can add these as new variables in the input_variables list. However, you also need to modify the prompt_template string to include placeholders for these new variables.

Here's an example of how you can do this:

prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.

{context}

Vector Store Output: {vector_store_output}

API Response: {api_response}

Question: {question}
Helpful Answer:"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "vector_store_output", "api_response", "question"]
)

In this example, {vector_store_output} and {api_response} are placeholders for the output of the vector stores and the standard API response, respectively. These will be replaced with the actual values when the prompt is generated.

You can find more information about how PromptTemplate handles multiple input variables in the LangChain documentation.

I hope this helps! Let me know if you have any other questions.

Sources

• libs/langchain/langchain/chains/retrieval_qa/prompt.py
• libs/langchain/langchain/prompts/prompt.py
• libs/langchain/langchain/schema/prompt_template.py
• libs/langchain/tests/unit_tests/prompts/test_prompt.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[chank20]

@dosu_bot how would I pass in the "vector_store_output" and "api_response" variables into the RetrievalQA chain called qa_chain?

[devstein]

Hey @chank20 as of now Dosu only responds to the discussion author. We're thinking of ways to make it easy for others to "fork" a conversation for their own needs.

If you have any ideas or feedback about Dosu, let me know!