chat keep repeating

[linqingfan]

Hi,
I m trying to use huggingface model TheBloke/OpenOrca-Platypus2-13B-GGML
I managed to create chromadb for external knowledge ingestion and uses conversationalretrievalchain
It works but I often encounter repeatition of self question and answer

I suspect it maybe incorrect prompt. The model template is

Instruction:

{prompt}

Response:

And I try different combinations of prompt template but I still have the same problems. One of the prompt template I use is:

prompt_template = """Answer based on context

{chat_history}

Question: {question}"""

Can anyone help?
regards
Ken

[dosubot[bot]]

🤖

Hi Ken,

Based on the information you've provided, it seems like you're trying to modify the prompt used in the ConversationalRetrievalChain. The default prompt used in this chain is CONDENSE_QUESTION_PROMPT, which is used to condense the chat history and new question into a standalone question for the retrieval step.

If you want to change this prompt, you can do so by passing it in via combine_docs_chain_kwargs if you instantiate the chain using from_llm. Here's an example of how you can do this:

ConversationalRetrievalChain.from_llm(combine_docs_chain_kwargs={"prompt": your_prompt})

Alternatively, you can use load_qa_chain with memory and add a custom prompt. Here's an example of how you can do this:

from langchain.chains.question_answering import load_qa_chain

template = """
  {Your_Prompt}
  
  CONTEXT:
  {context}
  
  QUESTION: 
  {query}

  CHAT HISTORY: 
  {chat_history}
  
  ANSWER:
  """

prompt = PromptTemplate(input_variables=["chat_history", "query", "context"], template=template)
memory = ConversationBufferMemory(memory_key="chat_history", input_key="query")

chain = load_qa_chain(ChatOpenAI(temperature=0), chain_type="stuff", memory=memory, prompt=prompt)

In this example, Your_Prompt should be replaced with the prompt you want to use. The context, query, and chat_history are placeholders for the actual context, query, and chat history.

These solutions were suggested in a similar issue in the LangChain repository: Issue: Changing Prompt (from Default) when Using ConversationalRetrievalChain?

If you're still encountering issues after trying these solutions, could you please provide more details about the problem? Specifically, it would be helpful to know more about the exact error message you're seeing and the steps you're taking that lead to this error.

Best,
Dosu

Sources

• Issue: Changing Prompt (from Default) when Using ConversationalRetrievalChain?
• libs/langchain/langchain/chains/conversational_retrieval/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[linqingfan]

Thanks Dosu, it can work now!