Multiple chains fails when there is more than one inputs

[vijayraghavanv]

makeChain = (retriever) => {
    const questionTemplate = PromptTemplate.fromTemplate(
      `Given the user question and the type below, rephrase the question by expanding any abbreviations.Respond with only the rephrased question and nothing else. Do not add any other words here.<question>{question}</question><type>{type}</type>formattedQuestion:`,
    );
    const answerTemplate = PromptTemplate.fromTemplate(
      `You are an expert in the field specified by the type {type}. Answer the provided question {formattedQuestion} using only the documents and context attached. If you don't know the answer, just say that you don't know, don't try to make up an answer. If the question is not related to the context or chat history, politely respond that you are tuned to only answer questions that are related to the context. <context>{context}</context>Answer:`,
    );

    const llm = new ChatOpenAI({
      temperature: 0, // increase temperature to get more creative answers
      modelName: 'gpt-4-0125-preview',
      openAIApiKey: 'key',
    });
    const questionGeneratorChain = questionTemplate
      .pipe(llm)
      .pipe(new StringOutputParser());
    const answerChain = RunnableSequence.from([
      {
        context: retriever.pipe(formatDocumentsAsString),
      },
      answerTemplate,
      llm,
      new StringOutputParser(),
    ]);
    const combinedChain = RunnableSequence.from([
      RunnablePassthrough.assign({ formattedQuestion: questionGeneratorChain }),
      answerChain,
    ]);
    return combinedChain;
  };

In the above code, I'm trying to combine two chains. The first chain is the questionGeneratorChain that provides a formattedQuestion, This should function as an input to the second chain(answerChain) along with the "type" that was provided while invoking the chain. So:

1. User invokes the combinedChain like so: chain.invoke ({question: 'My question', type: 'My type'}).
2. The questionGeneratorChain combines these two and produces a question.
3. This formattedQuestion along with 'type' should be passed to the answerChain which then uses PGVectorStore to retrieve the documents and generates the answer.
   However the issue is that the input to embedQuery (That is called internally ) is an object and is of shape: {question: 'My Question', type: "My type', formattedQuestion: 'The formatted question returned by question generator'}. EmbedQuery expects a text and not an object and this fails.
   How do I overcome this? I would prefer using RunnableSequences and not ConversationalQAChain.

[dosubot[bot]]

🤖

Hey @vijayraghavanv! Great to see you back here with another intriguing puzzle. How's it going?

Based on your code and the issue you're facing, it seems like the problem arises when the EmbedQuery function receives an object instead of a text. This is because the EmbedQuery function is designed to work with text inputs, not objects.

To resolve this, you can modify your RunnableSequence to ensure that the input to EmbedQuery is a string. You can achieve this by adding a transformation step in your RunnableSequence that extracts the required text from the input object and passes it to EmbedQuery.

Here's an example of how you can do this:

const transformInput = new Transform({
  transform: (input) => {
    return input.formattedQuestion; // or any other property that contains the text
  },
});

const answerChain = RunnableSequence.from([
  {
    context: retriever.pipe(formatDocumentsAsString),
  },
  answerTemplate,
  transformInput, // add this before llm
  llm,
  new StringOutputParser(),
]);

In this code, transformInput is a new Transform instance that extracts the text from the input object. This transform step is added to the answerChain before the llm step. This ensures that the input to EmbedQuery is a string, not an object.

Please note that you need to replace input.formattedQuestion with the actual property that contains the text you want to pass to EmbedQuery.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

For more information on how to use Transform and RunnableSequence, you can refer to the LangChainJS documentation.

Sources

• langchain/src/chains/router/multi_retrieval_qa.ts
• libs/langchain-cohere/src/embeddings.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[vijayraghavanv]

@dosu-bot : I dont see any Transform class. Am I missing something?

[dosubot[bot]]

🤖

In the LangChainJS framework, you can add a transformation step by creating a custom class that implements the Runnable interface. This class should have a run method that performs the transformation. Here's how you can modify the code to add a transformation step that extracts the required text from the input object and passes it to EmbedQuery:

import { ChatOpenAI, EmbedQuery } from "@langchain/openai";
import { StringOutputParser, Runnable, Stream } from "@langchain/core";

class TextExtractor implements Runnable {
  async run(input: any, stream: Stream): Promise<void> {
    const text = input.message; // replace this with your actual extraction logic
    await stream.push(new EmbedQuery(text));
  }
}

const chain = RunnableSequence.from([
  new ChatOpenAI({ temperature: 0 }),
  new TextExtractor(),
  new StringOutputParser(),
]);

const stream = await chain.stream({ message: "Hello there!" });

for await (const chunk of stream) {
  console.log(chunk);
}

In this code, TextExtractor is a custom class that implements the Runnable interface. The run method of this class extracts the required text from the input object and pushes it to the stream as an EmbedQuery object. The RunnableSequence is then updated to include an instance of TextExtractor after ChatOpenAI and before StringOutputParser. The input to chain.stream is also updated to be an object with a message property, as this is what TextExtractor expects.

You can find more information about the Runnable interface and how to implement it in the LangChainJS documentation.

Regarding the structure of the input object that is causing the issue in the EmbedQuery function, it's not directly mentioned in the provided context. However, the outputProcessor method in the StructuredQueryOutputParser class takes an input object that matches the queryInputSchema:

const queryInputSchema = /* #__PURE__ */ z.object({
  query: /* #__PURE__ */ z
    .string()
    .describe("text string to compare to document contents"),
  filter: /* #__PURE__ */ z
    .string()
    .optional()
    .describe("logical condition statement for filtering documents"),
});

This schema suggests that the input object should have the following structure:

{
  query: string; // A text string to compare to document contents
  filter?: string; // An optional logical condition statement for filtering documents
}

This object is used in the outputProcessor method as follows:

async outputProcessor({
    query,
    filter,
  }: z.infer<typeof queryInputSchema>): Promise<StructuredQuery> {
    let myQuery = query;
    if (myQuery.length === 0) {
      myQuery = " ";
    }
    if (filter === "NO_FILTER" || filter === undefined) {
      return new StructuredQuery(query);
    } else {
      const parsedFilter = await this.queryTransformer.parse(filter);
      return new StructuredQuery(query, parsedFilter);
    }
  }

If there is an issue with the EmbedQuery function, it could be related to the structure or values of this input object. However, without more specific information about the error or the EmbedQuery function itself, it's impossible to provide a more precise answer. You can find more information about the StructuredQueryOutputParser class and the queryInputSchema in the LangChainJS documentation.

Sources

• examples/src/prompts/string_output_parser_sequence.ts
• langchain/src/chains/query_constructor/index.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.