Can you tell me how to combine LLMChain with a RetrievalQChain?

[Elindo586]

I understand that the loadQAStuffChain function is used to initialize the LLMChain with a custom prompt template.???

^^^ Can you give me an example of how to use the LoadQAStuffChain function to use it in the LLMChain custom prompt template?

Right now the code I have doesn't work to combine LLMChain with RetrievalQAChain

Code:

import { TextLoader } from "langchain/document_loaders/fs/text";
import { CharacterTextSplitter } from "langchain/text_splitter";
import { OpenAIEmbeddings } from "langchain/embeddings/openai";
import { FaissStore } from "langchain/vectorstores/faiss";

import { OpenAI } from "langchain/llms/openai";
import { PromptTemplate } from "langchain/prompts";
import {
  SimpleSequentialChain,
  LLMChain,
  RetrievalQAChain,
  loadQAStuffChain,
} from "langchain/chains";

let loader;
let docs;
let splitter;
let documents;
let embeddings1;
let vectorstore;

let template1;
let promptTemplate1;
let LLMChain1;

let model1;
let model2;
let embeddings;
let vectoreStore;
let vectorChain;
let overallChain;

const useStore = async (req, res) => {
  if (req.method === "POST") {
    loader = new TextLoader("./documents/restaurant.txt");
    docs = await loader.load();
    splitter = new CharacterTextSplitter({
      chunkSize: 200,
      chunkOverlap: 50,
    });
    documents = await splitter.splitDocuments(docs);
    embeddings1 = new OpenAIEmbeddings();
    vectorstore = await FaissStore.fromDocuments(documents, embeddings1);
    await vectorstore.save("./");

    const message = req.body.newText;
    const firstOne = req.body.firstMsg;

    console.log(message);
    if (firstOne) {
      model1 = new OpenAI({
        openAIApiKey: process.env.OPENAI_API_KEY,
        temperature: 0,
      });
      template1 = `
    Be very funny when answering questions.
  {input}
    `;
      promptTemplate1 = new PromptTemplate({
        template: template1,
        inputVariables: ["input"],
      });

      LLMChain1 = new LLMChain({
        llm: model1,
        prompt: promptTemplate1,
      });

      model2 = new OpenAI({
        openAIApiKey: process.env.OPENAI_API_KEY,
        temperature: 0,
      });

      embeddings = new OpenAIEmbeddings();
      vectoreStore = await FaissStore.load("./", embeddings);

      vectorChain = new RetrievalQAChain({
        combineDocumentsChain: loadQAStuffChain(model2),
        retriever: vectoreStore.asRetriever(),
        returnSourceDocuments: false,
      });
    }

    overallChain = new SimpleSequentialChain({
      chains: [vectorChain, LLMChain1],
      verbose: true,
    });

    const result = await overallChain.run({
      input: message,
    });

    // const result = await LLMChain1.call({
    //   input: message,
    // });

    // const result = await vectorChain.call({
    //   query: message,
    // });

    console.log(result);

    return res.status(200).json({ output: result });
  } else {
    res.status(405).json({ say: "Only POST is allowed" });
  }
};
export default useStore;

[dosubot[bot]]

🤖

Hello,

From your code, it seems like you're trying to combine LLMChain with RetrievalQAChain using the loadQAStuffChain function. However, the loadQAStuffChain function is not designed to be used in this way.

The loadQAStuffChain function is used to create and load a StuffQAChain instance based on the provided parameters. It takes an instance of BaseLanguageModel and an optional StuffQAChainParams object as parameters. The StuffQAChainParams object can contain a prompt of type BasePromptTemplate and a verbose boolean flag.

Here is the function code for reference:

export function loadQAStuffChain(
  llm: BaseLanguageModel,
  params: StuffQAChainParams = {}
) {
  const { prompt = QA_PROMPT_SELECTOR.getPrompt(llm), verbose } = params;
  const llmChain = new LLMChain({ prompt, llm, verbose });
  const chain = new StuffDocumentsChain({ llmChain, verbose });
  return chain;
}

(Source: langchain/src/chains/question_answering/load.ts)

To correctly use the LLMChain with a RetrievalQAChain, you should use the RetrievalQAChain.fromLLM function. This function creates a RetrievalQAChain that uses the OpenAI LLM and the vector store retriever. Here is a code snippet that demonstrates this:

import { OpenAI } from "langchain/llms/openai";
import { RetrievalQAChain } from "langchain/chains";
import { HNSWLib } from "langchain/vectorstores/hnswlib";
import { OpenAIEmbeddings } from "langchain/embeddings/openai";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import * as fs from "fs";

// Initialize the LLM to use to answer the question.
const model = new OpenAI({});
const text = fs.readFileSync("state_of_the_union.txt", "utf8");
const textSplitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000 });
const docs = await textSplitter.createDocuments([text]);

// Create a vector store from the documents.
const vectorStore = await HNSWLib.fromDocuments(docs, new OpenAIEmbeddings());

// Initialize a retriever wrapper around the vector store
const vectorStoreRetriever = vectorStore.asRetriever();

// Create a chain that uses the OpenAI LLM and HNSWLib vector store.
const chain = RetrievalQAChain.fromLLM(model, vectorStoreRetriever);
const res = await chain.call({
  query: "What did the president say about Justice Breyer?",
});
console.log({ res });

(Source: examples/src/use_cases/local_retrieval_qa/qa_chain.ts)

I hope this helps! If you have any other questions, feel free to ask.

Sources

• examples/src/use_cases/local_retrieval_qa/qa_chain.ts
• langchain/src/chains/question_answering/load.ts
• examples/src/chains/retrieval_qa_custom.ts
• examples/src/chains/retrieval_qa_custom_prompt.ts
• examples/src/chains/retrieval_qa.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.