How to combine an LLMChain with a RetrievalQAChain into a SimpleSquentialChain?

[Elindo586]

Right now I can get the LLMChain or the RetrievalQAChain working separately.. but not together.

It seems like the RetrievalQAChain over takes the LLMChain and the answers are not logical...

Example:
Q. Who is the president of the United States?
A. Yes, of course. What vegetarian and vegan options are available at your restaurant?

How can I make these 2 chains work together?

Here is the code:

import { TextLoader } from "langchain/document_loaders/fs/text";
import { CharacterTextSplitter } from "langchain/text_splitter";
import { OpenAIEmbeddings } from "langchain/embeddings/openai";
import { FaissStore } from "langchain/vectorstores/faiss";

import { OpenAI } from "langchain/llms/openai";
import { PromptTemplate } from "langchain/prompts";
import {
  SimpleSequentialChain,
  LLMChain,
  RetrievalQAChain,
  loadQAStuffChain,
} from "langchain/chains";

let loader;
let docs;
let splitter;
let documents;
let embeddings1;
let vectorstore;

let template1;
let reviewPromptTemplate1;
let LLMChain1;

let model1;
let model2;
let embeddings;
let vectoreStore;
let vectorChain;
let overallChain;

const useStore = async (req, res) => {
  if (req.method === "POST") {
    loader = new TextLoader("./documents/restaurant.txt");
    docs = await loader.load();
    splitter = new CharacterTextSplitter({
      chunkSize: 200,
      chunkOverlap: 50,
    });
    documents = await splitter.splitDocuments(docs);
    embeddings1 = new OpenAIEmbeddings();
    vectorstore = await FaissStore.fromDocuments(documents, embeddings1);
    await vectorstore.save("./");

    const message = req.body.newText;
    const firstOne = req.body.firstMsg;

    console.log(message);
    if (firstOne) {
      model1 = new OpenAI({
        openAIApiKey: process.env.OPENAI_API_KEY,
        temperature: 0,
      });
      template1 = `
    Be very funny when answering questions.
  {input}
    `;
      reviewPromptTemplate1 = new PromptTemplate({
        template: template1,
        inputVariables: ["input"],
      });

      LLMChain1 = new LLMChain({
        llm: model1,
        prompt: reviewPromptTemplate1,
      });

      model2 = new OpenAI({
        openAIApiKey: process.env.OPENAI_API_KEY,
        temperature: 0,
      });

      embeddings = new OpenAIEmbeddings();
      vectoreStore = await FaissStore.load("./", embeddings);

      vectorChain = new RetrievalQAChain({
        combineDocumentsChain: loadQAStuffChain(model2),
        retriever: vectoreStore.asRetriever(),
        returnSourceDocuments: false,
      });
    }

    overallChain = new SimpleSequentialChain({
      chains: [LLMChain1, vectorChain],
      verbose: true,
    });

    // const result = await LLMChain1.call({
    //   input: message,
    // });

    const result = await overallChain.run({
      input: message,
    });

    console.log(result);

    return res.status(200).json({ output: result });
  } else {
    res.status(405).json({ say: "Only POST is allowed" });
  }
};
export default useStore;

[dosubot[bot]]

🤖

Hello,

From your code, it seems like you're trying to combine an LLMChain with a RetrievalQAChain into a SimpleSequentialChain. However, you're experiencing an issue where the RetrievalQAChain seems to overtake the LLMChain, resulting in illogical answers.

The SimpleSequentialChain class in LangChain is designed to run a sequence of chains, where the output of one chain is fed directly into the next. This means that the order in which you provide the chains to the SimpleSequentialChain constructor matters. In your case, you're providing the LLMChain first and then the RetrievalQAChain. This means that the LLMChain will run first, and its output will be used as the input for the RetrievalQAChain.

If the RetrievalQAChain is overtaking the LLMChain, it could be because the LLMChain is not producing a meaningful output that can be used as input for the RetrievalQAChain. This could be due to the way you've set up your LLMChain or the prompt template you're using.

Here's a simplified example of how you can set up a SimpleSequentialChain with an LLMChain and a RetrievalQAChain:

const llmChain = new LLMChain({
  llm: model1,
  prompt: reviewPromptTemplate1,
});

const retrievalQAChain = new RetrievalQAChain({
  combineDocumentsChain: loadQAStuffChain(model2),
  retriever: vectoreStore.asRetriever(),
  returnSourceDocuments: false,
});

const sequentialChain = new SimpleSequentialChain({
  chains: [llmChain, retrievalQAChain],
  verbose: true,
});

In this example, the llmChain is set up with a model and a prompt template. The retrievalQAChain is set up with a combineDocumentsChain (which is another LLMChain), and a retriever. The SimpleSequentialChain is then set up with the llmChain and the retrievalQAChain in that order.

Please note that the actual usage and implementation may vary based on the actual LangChain framework. The loadQAStuffChain function is used to initialize the LLMChain with a custom prompt template.

If you're still experiencing issues, it would be helpful if you could provide more information about how you're setting up your LLMChain and RetrievalQAChain, and what kind of output you're expecting.

I hope this helps! Let me know if you have any other questions.

Sources

• SimpleSequentialChain
• examples/src/chains/retrieval_qa.ts
• examples/src/use_cases/local_retrieval_qa/qa_chain.ts
• examples/src/chains/retrieval_qa_custom.ts
• examples/src/chains/retrieval_qa_with_remote.ts
• examples/src/chains/retrieval_qa_custom_prompt.ts
• langchain/src/chains/sequential_chain.ts
• langchain/src/chains/tests/simple_sequential_chain.int.test.ts
• langchain/src/chains/tests/simple_sequential_chain.test.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.