How am I able to give a unique identifier to the embeddings when they get created? Using ConvexVectorStore

[achris-alonzo30]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

// Code for creating the embeddings and storing it Convex Vectore Database.

export const createEmbeddings = internalAction({
args: {
storageFileId: v.id("_storage"),
},
handler: async (ctx, args) => {
let fileUrl = await ctx.storage.get(args.storageFileId);

    const loader = new PDFLoader(fileUrl ?? "");

    if (!loader) console.log("Failed to create loader");

    const docs = await loader.load();

    const textSplitter = new RecursiveCharacterTextSplitter({
        chunkSize: 1000,
        chunkOverlap: 200
    });

    const splitDocs = await textSplitter.splitDocuments(docs);

    // ctx.scheduler.runAfter(0, internal.ingest.load. )

    
    const embeddings = new CacheBackedEmbeddings({
        underlyingEmbeddings: new OpenAIEmbeddings(),
        documentEmbeddingStore: new ConvexKVStore({  ctx }),
    });

    // Store the embeddings in the convex database
    await ConvexVectorStore.fromDocuments(splitDocs, embeddings, { ctx })
}

})

// Code for creating the answer of user question

export const answer = internalAction({
args: {
sessionId: v.string(),
message: v.string(),
fileId: v.id("_storage"),
},
handler: async (ctx, args) => {
const vectorStore = new ConvexVectorStore(new OpenAIEmbeddings(), { ctx });
const model = new ChatOpenAI({ modelName: OPENAI_MODEL, temperature: 0 });

const prompt = ChatPromptTemplate.fromTemplate(
  `Important: You are an intelligent chatbot designed to help users by answering questions only on Enterprise services & activities.
  Answer the question only if there is information in the chat history (delimited by []) and context (delimited by {}) below.
  If context is not empty and the answer cannot be determined from context, say "I cannot determine the answer from context". If you don't know the answer, just say that you don't know, don't try to make up an answer. Do not print your answer starting with "Answer:"

  (You do not need to use these pieces of information if not relevant)

  Current conversation:
  Human: {args.message}
  AI:`
);

// Retrieves the chat history from a single conversataion
const memory = new BufferMemory({
  chatHistory: new ConvexChatMessageHistory({
    sessionId: args.sessionId,
    ctx,
  }),
  memoryKey: `chat_history`,
  inputKey: `question`,
});

const chain = ConversationalRetrievalQAChain.fromLLM(
  model,
  vectorStore.asRetriever(),
  {
    memory,
    verbose: true,
  }
);

await chain.invoke({ question: args.message, prompt, fileId: args.fileId });

},
});

Description

Use Case
I'm trying to find a way to add a unique key when I store the embeddings in the ConvexVectorStore, if that is possible?

Problem

• I'm encountering an issue when I load a new document to start a new conversation the LLM still retains the answer from the previous document.

• I also found when it try to search the embeddings to find the answer it also fetched the embeddings from the previous documents, I'm not sure if this what should happen?

Expectation

• I expect to be able to isolate and filter the chat history as well the embeddings that is being fetched when the LLM look for the document.
• I don't want LLM to answer using the previous documents and should only focus to the current document that is loaded. -- Then if I try to go back to the same document it should load the previous document as well retain the previous message within that document,

System Info

Dependencies

"dependencies": {
"@clerk/clerk-sdk-node": "^4.13.11",
"@clerk/nextjs": "^4.29.9",
"@hookform/resolvers": "^3.3.4",
"@instructor-ai/instructor": "^0.0.7",
"@langchain/openai": "^0.0.18",
"@langchain/pinecone": "^0.0.3",
"@pinecone-database/doc-splitter": "^0.0.1",
"@pinecone-database/pinecone": "^2.1.0",
"@radix-ui/react-accordion": "^1.1.2",
"@radix-ui/react-alert-dialog": "^1.0.5",
"@radix-ui/react-avatar": "^1.0.4",
"@radix-ui/react-dialog": "^1.0.5",
"@radix-ui/react-dropdown-menu": "^2.0.6",
"@radix-ui/react-icons": "^1.3.0",
"@radix-ui/react-label": "^2.0.2",
"@radix-ui/react-progress": "^1.0.3",
"@radix-ui/react-select": "^2.0.0",
"@radix-ui/react-slot": "^1.0.2",
"@radix-ui/react-tabs": "^1.0.4",
"@radix-ui/react-toast": "^1.1.5",
"@radix-ui/react-tooltip": "^1.0.7",
"@tanstack/react-table": "^8.13.2",
"ai": "^3.0.7",
"class-variance-authority": "^0.7.0",
"clsx": "^2.1.0",
"convex": "^1.10.0",
"date-fns": "^3.3.1",
"framer-motion": "^11.0.8",
"i": "^0.3.7",
"langchain": "^0.1.25",
"lucide-react": "^0.344.0",
"next": "14.1.1",
"next-themes": "^0.2.1",
"npm": "^10.5.0",
"openai": "^4.28.4",
"pdf-parse": "^1.1.1",
"react": "^18",
"react-dom": "^18",
"react-hook-form": "^7.51.0",
"react-loader-spinner": "^6.1.6",
"react-loading-skeleton": "^3.4.0",
"react-markdown": "^9.0.1",
"react-pdf": "^7.7.1",
"react-resize-detector": "^10.0.1",
"react-textarea-autosize": "^8.5.3",
"react-wrap-balancer": "^1.1.0",
"replicate": "^0.18.1",
"simplebar-react": "^3.2.4",
"svix": "^1.20.0",
"tailwind-merge": "^2.2.1",
"tailwindcss-animate": "^1.0.7",
"zod": "^3.22.4"
},
"devDependencies": {
"@types/node": "^20",
"@types/react": "^18",
"@types/react-dom": "^18",
"autoprefixer": "^10.0.1",
"eslint": "^8",
"eslint-config-next": "14.1.1",
"postcss": "^8",
"tailwindcss": "^3.3.0",
"typescript": "^5"
}

Platform
Mac

[dosubot[bot]]

Hey @achris-alonzo30! Great to see you back again. I hope you're doing wonderfully. 😊

Yes, there is a way to assign a unique key to the embeddings when they are stored in the ConvexVectorStore to isolate and filter the chat history and embeddings for each document. This can be achieved by modifying the namespace or key generation strategy used in the CacheBackedEmbeddings configuration. In the provided context, the namespace is set to underlyingEmbeddings.modelName, which means all embeddings are stored under a common namespace derived from the model name.

To isolate embeddings per document, you can append a unique identifier for each document to the namespace or modify the key generation logic to include document-specific information. This unique identifier could be a hash of the document's content, a UUID, or any other unique attribute associated with the document. By doing so, the Language Learning Model (LLM) can be directed to use only the embeddings associated with the current document's unique key, ensuring it does not retain or use answers from previous documents.

Here's an illustrative modification to include a document-specific identifier in the namespace:

// Assuming each document has a unique identifier, e.g., a hash or UUID
const documentId = "unique_document_identifier"; // This should be dynamically determined for each document

const cacheBackedEmbeddings = CacheBackedEmbeddings.fromBytesStore(
  underlyingEmbeddings,
  inMemoryStore,
  {
    namespace: `${underlyingEmbeddings.modelName}-${documentId}`,
  }
);

In this modification, documentId is appended to the namespace, ensuring that embeddings are stored and retrieved in a way that is isolated per document. This approach requires that you manage the document identifiers and ensure they are correctly assigned and used when storing and retrieving embeddings for each document.

The CacheBackedEmbeddings class in LangChain JS handles key generation for storing embeddings by using the insecureHash function to hash the document content, potentially prefixed with a namespace if provided. This is evident in the fromBytesStore static method, where the keyEncoder function is defined as (key) => (options?.namespace ?? "") + insecureHash(key). This means that the key for storing embeddings is generated by first checking if a namespace option is provided and using it as a prefix (defaulting to an empty string if not provided), and then appending the hash of the document content generated by the insecureHash function.

You can find more details in the LangChain JS source code:

• examples/src/embeddings/cache_backed_in_memory.ts
• langchain/src/embeddings/cache_backed.ts

Please note that this approach assumes that you have a mechanism to manage and assign unique identifiers to each document. If you don't have such a mechanism in place, you might need to implement one. Also, keep in mind that changing the namespace or key generation strategy will affect how embeddings are stored and retrieved, so you should thoroughly test any changes to ensure they meet your requirements.

Sources

• examples/src/embeddings/cache_backed_in_memory.ts
• langchain/src/embeddings/cache_backed.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[achris-alonzo30]

Okay, now how am I able to make sure to filter out that particular document using the namespace I provided, when the LLM answer and looks for the documents in the vector store to answer the user question? because I want to isolate the memory and the messages for each particular document, so when user ask a question the LLM doesn't use any documents or answer from the previous documents, only the document that is loaded.

`
export const answer = internalAction({
args: {
sessionId: v.string(),
message: v.string(),
fileId: v.id("_storage"),
},
handler: async (ctx, args) => {
const vectorStore = new ConvexVectorStore(new OpenAIEmbeddings(), { ctx });
const model = new ChatOpenAI({ modelName: OPENAI_MODEL, temperature: 0 });

const prompt = ChatPromptTemplate.fromTemplate(
  `Important: You are an intelligent chatbot designed to help users by answering questions only on Enterprise services & activities.
  Answer the question only if there is information in the chat history (delimited by []) and context (delimited by {}) below.
  If context is not empty and the answer cannot be determined from context, say "I cannot determine the answer from context". If you don't know the answer, just say that you don't know, don't try to make up an answer. Do not print your answer starting with "Answer:"

  (You do not need to use these pieces of information if not relevant)

  Current conversation:
  Human: {args.message}
  AI:`
);

// Retrieves the chat history from a single conversataion
const memory = new BufferMemory({
  chatHistory: new ConvexChatMessageHistory({
    sessionId: args.sessionId,
    ctx,
  }),
  memoryKey: `chat_history`,
  inputKey: `question`,
});

const chain = ConversationalRetrievalQAChain.fromLLM(
  model,
  vectorStore.asRetriever(),
  {
    memory,
    verbose: true,
  }
);

await chain.invoke({ question: args.message, prompt, fileId: args.fileId });

},
});
`