How to add specific source from db to RunnableSequence using Langchain

[teomikitenko]

import { v } from "convex/values";
import { ChatOpenAI } from "@langchain/openai";
import { OpenAIEmbeddings } from "@langchain/openai";
import { ConvexVectorStore } from "@langchain/community/vectorstores/convex";
import { action } from "../_generated/server";
import { StringOutputParser } from "@langchain/core/output_parsers";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import {
  RunnableSequence,
  RunnablePassthrough,
} from "@langchain/core/runnables";
import { pull } from "langchain/hub";
import { formatDocumentsAsString } from "langchain/util/document";
import { fetchAction, fetchQuery,fetchMutation } from "convex/nextjs";
import { api, internal } from "../_generated/api";
import { Doc } from "../_generated/dataModel";



export const answer = action({
  args: {
    question: v.string(),
  },
  handler: async (ctx, { question }) => {
    const embeddings = new OpenAIEmbeddings()
    const res = await ctx.vectorSearch('documents','byEmbedding',{
    vector: await embeddings.embedQuery(question)
    })
    const searchResult:Doc<'documents'> = await ctx.runQuery(internal._search.search.searchDocument,{id:res[0]._id})
     const vectorStore = new ConvexVectorStore(new OpenAIEmbeddings(), { ctx });
    const llm = new ChatOpenAI({
      apiKey: process.env.OPENAI_API_KEY,
      model: "gpt-3.5-turbo",
      temperature: 0,
    });
    const parcer = new StringOutputParser();
    const retriever = vectorStore.asRetriever()
    const prompt = await pull<ChatPromptTemplate>("rlm/rag-prompt");
    const ragChain = RunnableSequence.from([
      {
        context: retriever.pipe(formatDocumentsAsString) ,
        question: new RunnablePassthrough(),
      },
      prompt,
      llm,
      parcer,
    ]);
    const stream = await ragChain.stream(question);
    return stream; 
  },
});

I use convex and langchain to create my rag application. In my case i need add to context specific data from my db - logic my application is serching some embeddings by id from my vectoredb then i use its embeddings for generate answer - but if i just add to context from RunnableSequence my embeddings i need use some format or embeddings are enough for this? I dont find this sample in langchain documentation

using langchainjs in next.js
nodejs -v20.15.1
windows 10

[dosubot[bot]]

Hey there, @teomikitenko! I'm here to assist you with any issues you encounter. Let's get started!

To add specific data from your database to the context in a RunnableSequence using LangChain and Convex, you can follow these steps:

1. Query the database to get the specific data:

   let properNouns: string[] = await queryAsList(db, "SELECT Name FROM Artist");
   properNouns = properNouns.concat(
     await queryAsList(db, "SELECT Title FROM Album")
   );
   properNouns = properNouns.concat(
     await queryAsList(db, "SELECT Name FROM Genre")
   );

2. Embed and store the values in a vector database:

   const vectorDb = await MemoryVectorStore.fromTexts(
     properNouns,
     {},
     new OpenAIEmbeddings()
   );
   const retriever = vectorDb.asRetriever(15);

3. Create a RunnableSequence to retrieve values and insert them into the context:

   const formatDocs = (docs: Document[]) => docs.map((doc) => doc.pageContent);
   const retrieverChain = RunnableSequence.from([
     { context: retriever.pipe(formatDocs), question: new RunnablePassthrough() },
   ]);

4. Assign the retrieved values to the context and pipe it to the query chain:

   const chain = RunnablePassthrough.assign({
     proper_nouns: retrieverChain,
   }).pipe(queryChain);

Here is the complete code snippet for reference:

import { DocumentInterface } from "@langchain/core/documents";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import {
  RunnablePassthrough,
  RunnableSequence,
} from "@langchain/core/runnables";
import { ChatOpenAI, OpenAIEmbeddings } from "@langchain/openai";
import { createSqlQueryChain } from "langchain/chains/sql_db";
import { SqlDatabase } from "langchain/sql_db";
import { MemoryVectorStore } from "langchain/vectorstores/memory";
import { DataSource } from "typeorm";

const datasource = new DataSource({
  type: "sqlite",
  database: "../../../../Chinook.db",
});
const db = await SqlDatabase.fromDataSourceParams({
  appDataSource: datasource,
});

async function queryAsList(database: any, query: string): Promise<string[]> {
  const res: Array<{ [key: string]: string }> = JSON.parse(
    await database.run(query)
  )
    .flat()
    .filter((el: any) => el != null);
  const justValues: Array<string> = res.map((item) =>
    Object.values(item)[0]
      .replace(/\b\d+\b/g, "")
      .trim()
  );
  return justValues;
}

let properNouns: string[] = await queryAsList(db, "SELECT Name FROM Artist");
properNouns = properNouns.concat(
  await queryAsList(db, "SELECT Title FROM Album")
);
properNouns = properNouns.concat(
  await queryAsList(db, "SELECT Name FROM Genre")
);

const vectorDb = await MemoryVectorStore.fromTexts(
  properNouns,
  {},
  new OpenAIEmbeddings()
);
const retriever = vectorDb.asRetriever(15);

const system = `You are a SQLite expert. Given an input question, create a syntactically correct SQLite query to run.
Unless otherwise specified, do not return more than {top_k} rows.
Here is the relevant table info: {table_info}

Here is a non-exhaustive list of possible feature values.
If filtering on a feature value make sure to check its spelling against this list first:

{proper_nouns}`;
const prompt = ChatPromptTemplate.fromMessages([
  ["system", system],
  ["human", "{input}"],
]);

const llm = new ChatOpenAI({ model: "gpt-4", temperature: 0 });

const queryChain = await createSqlQueryChain({
  llm,
  db,
  prompt,
  dialect: "sqlite",
});
const formatDocs = (docs: Document[]) => docs.map((doc) => doc.pageContent);
const retrieverChain = RunnableSequence.from([
  { context: retriever.pipe(formatDocs), question: new RunnablePassthrough() },
]);
const chain = RunnablePassthrough.assign({
  proper_nouns: retrieverChain,
}).pipe(queryChain);

const query2 = await chain.invoke({
  question: "What are all the genres of Elenis Moriset songs?",
});
console.log("query2", query2);
console.log("db query results", await db.run(query2));

This code demonstrates how to add specific data from your database to the context in a RunnableSequence using LangChain and Convex. The RunnablePassthrough class is used to pass data through the chain unchanged, ensuring that the proper nouns retrieved from the database are correctly formatted and included in the context for the query chain ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.

[teomikitenko]

@dosu pls give example with convex db