How return id from ConvexVectorStore.fromDocuments

[teomikitenko]

import { v } from "convex/values";
import { action } from "../_generated/server";
import { Document } from "@langchain/core/documents";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { CacheBackedEmbeddings } from "langchain/embeddings/cache_backed";
import { ConvexKVStore } from "@langchain/community/storage/convex";
import { ConvexVectorStore } from "@langchain/community/vectorstores/convex";
import { OpenAIEmbeddings } from "@langchain/openai";

export const loadEmbeddings = action({
  args: {
    text: v.string(),
  },
  handler: async (ctx, { text }) => {
    const textSplitter = new RecursiveCharacterTextSplitter({
      chunkSize: 1000,
      chunkOverlap: 200,
    });
    const splitDocs = await textSplitter.splitDocuments([
      new Document({ pageContent: text }),
    ]);

    const embeddings = new CacheBackedEmbeddings({
      underlyingEmbeddings: new OpenAIEmbeddings(),
      documentEmbeddingStore: new ConvexKVStore({ ctx }),
    });
    await ConvexVectorStore.fromDocuments(splitDocs, embeddings, { ctx });
  },
});

_I write embedding using Convex const vectoreStore = await ConvexVectorStore.fromDocuments(splitDocs, embeddings, { ctx });how do I determine the ID provided by the database for this embed . My application use this ID for search current embeddings - but i dont see information about this in documentation. How i do this using Convex Vectore Store

using LangchainJS in Next.js and Convex
nodejs -v20.15.1
windows 10_

[dosubot[bot]]

To determine the ID provided by the database for an embedding when using ConvexVectorStore.fromDocuments, you can use the addDocuments method, which internally calls the addVectors method. The addVectors method returns the IDs of the added vectors. Here is the relevant code snippet:

/**
 * Method to add documents to the Xata database. Maps the page content of
 * each document, embeds the documents using the embeddings, and adds the
 * vectors to the database.
 * @param documents Array of documents to be added.
 * @param options Optional object containing an array of ids.
 * @returns Promise resolving to an array of ids of the added documents.
 */
async addDocuments(documents: Document[], options?: { ids?: string[] }) {
  const texts = documents.map(({ pageContent }) => pageContent);
  return this.addVectors(
    await this.embeddings.embedDocuments(texts),
    documents,
    options
  );
}

/**
 * Method to add vectors to the Xata database. Maps each vector to a row
 * with the document's content, embedding, and metadata. Creates or
 * replaces these rows in the Xata database.
 * @param vectors Array of vectors to be added.
 * @param documents Array of documents corresponding to the vectors.
 * @param options Optional object containing an array of ids.
 * @returns Promise resolving to an array of ids of the added vectors.
 */
async addVectors(
  vectors: number[][],
  documents: Document[],
  options?: { ids?: string[] }
) {
  const rows = vectors
    .map((embedding, idx) => ({
      content: documents[idx].pageContent,
      embedding,
      ...documents[idx].metadata,
    }))
    .map((row, idx) => {
      if (options?.ids) {
        return { id: options.ids[idx], ...row };
      }
      return row;
    });

  const res = await this.client.db[this.table].createOrReplace(rows);
  const results = res as unknown as { id: string }[];
  const returnedIds = results.map((row) => row.id);
  return returnedIds;
}

When you call fromDocuments, it will internally call addDocuments, which will return the IDs of the added vectors ^[1][2][3].

To continue talking to Dosu, mention @dosu.