Retrieve from multiple sources

[latifs]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

// ... more code above nothing relevant 

  const retrieverChain = RunnableSequence.from([
    (prevResult) => prevResult.standalone_question,
    vectorStore.asRetriever(),
    combineDocuments,
  ]);

  const chain = RunnableSequence.from([
    {
      standalone_question: standaloneQuestionChain,
      original_input: new RunnablePassthrough(),
    },
    {
      context: retrieverChain, // can I pass multiple retriever here: [retrieverChain, queryChain]
      question: ({ original_input }) => original_input.question,
      conv_history: ({ original_input }) => original_input.conv_history,
    },
    answerChain,
  ]);

  const response = await chain.invoke({
    question: question,
    conv_history: formatConvHistory(convHistory), // here we pass the conversation to our chain
  });

Description

Hey guys,
turns out that LLMs are terrible at getting information from a CSV file so I resorted to turning uploaded CSV files to sqlite files and use a text to Query to create a queryChain that will basically send a sql query to the db file.
but I'd still like to get data from the vector store too and combine the result of both to pass to the llm.
Do you have a simpler solution? am i wrong about CSV files?

System Info

Langchain

[dosubot[bot]]

Yes, it is possible to pass multiple retrievers to the context in the chain in LangChain JS. Your approach of combining results from the vector store and a queryChain created for a SQL query to a database file is a valid concept. However, the provided code does not directly show the integration with a SQL database or the use of a queryChain for SQL queries.

To achieve the combination of vector store results and SQL query results, you would need to extend the logic beyond what is shown. This could involve:

1. Creating a retriever that can execute SQL queries against your database file.
2. Combining the results from this SQL retriever with the MultiVectorRetriever results, either by merging the results based on some logic or by feeding the results of one into the other as a filtering or ranking step.

In LangChain JS, the MultiQueryRetriever handles multiple retrievers in a RunnableSequence by generating multiple queries from a single input query and then retrieving documents relevant to each of these generated queries. This approach allows for a more comprehensive retrieval of information by considering different ways of asking the same question, thereby increasing the chances of finding relevant documents.

For more details, you can refer to the following examples in the LangChain JS repository:

• MultiVectorRetriever example
• MultiQueryRetriever example

A simpler or more correct solution would depend on the specific requirements of your application, such as how you intend to combine the results and what criteria are used for retrieval from both sources.

Sources

• examples/src/retrievers/multi_vector_hypothetical.ts
• examples/src/retrievers/multi_query.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[latifs]

so you are right I didn't paste the code for the queryChain here but it is basically:

• a standalone question
• a text to SQL query where the question is turned to a query is then ran against the db file.
• then a result comes out

In the case where both the answer from the queryChain and the answer from the retrieverChain(the one that gets the data from a vector store) are both needed for the LLM to generate the answer what is the best approach in this particular scenario?