Can you please help me write a custom question generator chain prompt for my code to work?

[Elindo586]

I need to include a questionGeneratorChainOptions template for the bot to understand and properly answer follow up questions given the history.... could you help me write the correct prompt template for the code to work?

Code here:

import { ChatOpenAI } from "langchain/chat_models/openai";
import { ConversationalRetrievalQAChain } from "langchain/chains";
import { BufferMemory } from "langchain/memory";

import { TextLoader } from "langchain/document_loaders/fs/text";
import { CharacterTextSplitter } from "langchain/text_splitter";
import { OpenAIEmbeddings } from "langchain/embeddings/openai";
import { FaissStore } from "langchain/vectorstores/faiss";

let loader;
let docs;
let splitter;
let documents;
let embeddings1;
let vectorstore;

let model1;

let embeddings;
let vectoreStore;

let chain;
let result;
let chatHistory;
let followUpRes;

const useStore = async (req, res) => {
  if (req.method === "POST") {
    loader = new TextLoader("./documents/restaurant.txt");
    docs = await loader.load();
    splitter = new CharacterTextSplitter({
      chunkSize: 200,
      chunkOverlap: 50,
    });
    documents = await splitter.splitDocuments(docs);
    embeddings1 = new OpenAIEmbeddings();
    vectorstore = await FaissStore.fromDocuments(documents, embeddings1);
    await vectorstore.save("./");

    const message = req.body.newText;
    const firstOne = req.body.firstMsg;

    console.log(message);
    if (firstOne) {
      model1 = new ChatOpenAI({
        modelName: "gpt-4",
        openAIApiKey: process.env.OPENAI_API_KEY,
        temperature: 0,
      });

      embeddings = new OpenAIEmbeddings();
      vectoreStore = await FaissStore.load("./", embeddings);
    }

    const CUSTOM_QUESTION_GENERATOR_CHAIN_PROMPT = `Given the following conversation and a follow up question, return the conversation history excerpt that includes any relevant context to the question if it exists and rephrase the follow up question to be a standalone question.
Chat History:
{chat_history}
Follow Up Input: {question}
Your answer should follow the following format:
\`\`\`
Use the following pieces of context to answer the users question.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
----------------
{chatHistory}
Standalone question: {message}
\`\`\`
Your answer:`;

    chain = ConversationalRetrievalQAChain.fromLLM(
      model1,
      vectorstore.asRetriever(),
      {
        memory: new BufferMemory({
          memoryKey: "chat_history", // Must be set to "chat_history"
          inputKey: "question", // The key for the input to the chain
          outputKey: "text", // The key for the final conversational output of the chain
          returnMessages: true,
          questionGeneratorChainOptions: {
            template: CUSTOM_QUESTION_GENERATOR_CHAIN_PROMPT,
          },
        }),
      }
    );
    chatHistory = "";

    result = await chain.call({
      question: message,
      chat_history: chatHistory,
    });
    chatHistory = `${message}\n${result.text}`;

    followUpRes = await chain.call({
      question: message,
      chat_history: chatHistory,
    });

    console.log(result);
    console.log(followUpRes);

    return res.status(200).json({ output: result });
  } else {
    res.status(405).json({ say: "Only POST is allowed" });
  }
};
export default useStore;

[dosubot[bot]]

🤖

Sure, I can help you with that. The questionGeneratorChainOptions is an option in the ConversationalRetrievalQAChain.fromLLM method that allows you to specify a custom template for generating questions. In your case, you want to create a template that allows the bot to understand and respond to follow-up questions based on the conversation history.

Here's how you can do it:

const CUSTOM_QUESTION_GENERATOR_CHAIN_PROMPT = `Given the following conversation and a follow up question, return the conversation history excerpt that includes any relevant context to the question if it exists and rephrase the follow up question to be a standalone question.
Chat History:
{chat_history}
Follow Up Input: {question}
Your answer should follow the following format:
\`\`\`
Use the following pieces of context to answer the users question.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
----------------
<Relevant chat history excerpt as context here>
Standalone question: <Rephrased question here>
\`\`\`
Your answer:`;

const chain = ConversationalRetrievalQAChain.fromLLM(
  model,
  vectorStore.asRetriever(),
  {
    memory: new BufferMemory({
      memoryKey: "chat_history",
      returnMessages: true,
    }),
    questionGeneratorChainOptions: {
      template: CUSTOM_QUESTION_GENERATOR_CHAIN_PROMPT,
    },
  }
);

In the CUSTOM_QUESTION_GENERATOR_CHAIN_PROMPT template, {chat_history} will be replaced with the actual chat history and {question} will be replaced with the follow-up question. The bot will then use this template to generate a standalone question based on the conversation history and the follow-up question.

In the context shared, the ConversationalRetrievalQAChain class has a fromLLM static method that takes in a BaseLanguageModel, a BaseRetriever, and an options object. The options object can include questionGeneratorChainOptions which can have a template property. This template is used as the prompt for the LLMChain that generates the standalone question. If no template is provided, a default template is used. The fromLLM method returns a new instance of ConversationalRetrievalQAChain.

You can find more information about this in the examples/src/chains/conversation_qa_custom_prompt.ts and langchain/src/chains/conversational_retrieval_chain.ts files in the langchainjs repository.

I hope this helps! Let me know if you have any other questions.

Sources

• examples/src/chains/conversation_qa_custom_prompt.ts
• langchain/src/chains/conversational_retrieval_chain.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.