Prisma Vector Store's createDocuments does not work as expected

[phyziyx]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

This is a simplified example that I've used for debugging the issue:

model Document {
  // Could also use id as String of cuid()
  id        Int      @id @default(autoincrement())
  createdAt DateTime @default(now()) @db.Timestamp(0)

  // namespace String?                @default("default")
  content String
  vector  Unsupported("vector")?
  deckId  Int
}

// I'm trying to `addDocuments` however, it does not seem to add to the DB.

const { deckId } = req.params.body;

const vectorStore = new PrismaVectorStore(this.ollamaEmbeddings, {
	db: prisma,
	prisma: Prisma,
	tableName: "Document",
	vectorColumnName: "vector",
	columns: {
		id: PrismaVectorStore.IdColumn,
		content: PrismaVectorStore.ContentColumn,
	},
	filter: {
		deckId: {
			equals: Number(deckId),
		},
	},
});

const rawTexts = [
	"Hello World!",
	"Caleb's favourite fruit is Apple.",
	"Bye bye!",
];

// Split the raw texts into documents
const docs = await textSplitter.createDocuments(rawTexts);
console.log(`docs: ${docs[0].pageContent}`);

await vectorStore
	.addDocuments(docs)
	.then(() => console.log('Added'))
	.catch((e) => console.error(`Error: ${e}`);```

Description

1. Why is there no metadata column for Prisma? Is this an oversight?
2. I've resorted to using addModels and running a Prisma transaction as shown on the documentation here (https://js.langchain.com/docs/integrations/vectorstores/prisma). Is this the expected way to do it?
3. What is wrong in my example?

System Info

Node JS v21.7.2
Prisma ORM
Windows 10 x64
LangChain 0.2.0

[dosubot[bot]]

To address the issues you're encountering with createDocuments and the Prisma Vector Store in LangChain:

1. Metadata Column: The absence of a metadata column in your Prisma schema is by design, tailored to specific application needs. If additional metadata storage is required for each document, you should extend the Document model in your schema accordingly. Ensure any new fields are integrated into your application logic for document handling.

2. Using addModels vs. Expected Approach: The correct approach for adding documents involves the addDocuments method, which you're already using. This method is designed to handle document addition by generating vector representations for the document content and storing these vectors in the database. The use of addModels is not the standard method for adding documents with their vector representations in the Prisma Vector Store setup.

3. Troubleshooting Document Addition: The core issue might lie in how documents are being prepared and added. Ensure that the createDocuments method from your textSplitter is correctly generating Document instances from your text array. This step is crucial for the subsequent vector generation and storage process. It's not clear from your snippet how textSplitter is defined or if it's properly configured, which could be a factor in the problem.

To resolve the issue, verify that your document creation and addition process includes the necessary steps for embedding generation and vector storage. This might require adjusting your implementation to ensure that document vectors are correctly generated and stored alongside the document content in the database. For further guidance, consider reviewing the implementations of PrismaVectorStore and TokenTextSplitter in the LangChain JS repository for examples of correct usage and configuration.

Details

Sources

• examples/src/indexes/vector_stores/prisma_vectorstore/prisma.ts
• libs/langchain-community/src/vectorstores/prisma.ts
• examples/src/indexes/vector_stores/prisma_vectorstore/prisma/schema.prisma
• libs/langchain-community/src/embeddings/ollama.ts
• langchain/src/document_loaders/fs/text.ts
• examples/src/indexes/token_text_splitter.ts

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.