Is there a way to get the list of collection in langchain/community/vectorstores/chromadb?

[lalitx17]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import { LlamaCpp } from "@langchain/community/llms/llama_cpp";
import { HuggingFaceTransformersEmbeddings } from "@langchain/community/embeddings/hf_transformers";
import { Chroma } from "@langchain/community/vectorstores/chroma";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { createRetrievalChain } from "langchain/chains/retrieval";
import { Document } from "langchain/document";

const llamaPath = "/home/lalit/models/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf";

const embeddings = new HuggingFaceTransformersEmbeddings({
  modelName: "Xenova/all-MiniLM-L6-v2",
});

async function initChroma(collectionName: string) {
  return await Chroma.fromExistingCollection(embeddings, { collectionName });
}


async function getAllDocumentsFromCollection(store: Chroma): Promise<Document[]> {
  const result = await store.similaritySearch("", 10000);
  console.log(result);
  return result;
}

async function setupChain(query: string) {
//I HAVE TO ADD THE COLLECTIONS MANUALLY SINCE I COULDN'T FIND ANY INBUILT FUNCTIONS TO DO RETRIEVE IT.
  const collections = ["paul_graham_essays"];           
  const vectorStores = await Promise.all(
    collections.map(collectionName => initChroma(collectionName))
  );

  let allDocuments: Document[] = [];
  for (const store of vectorStores) {
    const docs = await getAllDocumentsFromCollection(store);
    allDocuments = allDocuments.concat(docs);
  }

 
  const tempVectorStore = await Chroma.fromDocuments(
    allDocuments,
    embeddings,
    { collectionName: "temp_collection" }
  );

  // Perform similarity search on all documents
  const searchResults = await tempVectorStore.similaritySearchWithScore(query, allDocuments.length);

  // Sort by similarity score (descending) and take top 50
  const topDocuments = searchResults
    .sort((a, b) => b[1] - a[1])
    .slice(0, 100)
    .map(item => item[0]);

  // Create a new combined vector store with top 50 documents
  const combinedVectorStore = await Chroma.fromDocuments(
    topDocuments,
    embeddings,
    { collectionName: "combined_collection" }
  );

  // Initialize LLaMa model
  const model = new LlamaCpp({
    modelPath: llamaPath,
    temperature: 0.7,
    maxTokens: 2000,
    topP: 0.3,
    contextSize: 4096,
  });

  // Create a custom prompt template
  const prompt = ChatPromptTemplate.fromTemplate(`
Use the following pieces of context to answer the question at the end. 
If you don't know the answer, just say that you don't know, don't try to make up an answer.

Context: {context}

Question: {question}

Please provide a detailed and comprehensive answer:
  `);

  // Create the combine documents chain
  const combineDocsChain = await createStuffDocumentsChain({
    llm: model,
    prompt,
  });

  // Create the retriever
  const retriever = combinedVectorStore.asRetriever();

  // Create the retrieval chain
  const retrievalChain = await createRetrievalChain({
    combineDocsChain,
    retriever,
  });

  return retrievalChain;
}



export async function queryChain(query: string) {
  const chain = await setupChain(query);
  const response = await chain.invoke({ input: query, question: query });
  
  return {
    text: response.answer,
    sourceDocuments: response.context,
  };
}

Description

I couldn't find a way to do it unless I use the chromadb package independently.

System Info

using langchainjs in next.js
linux fedora workstation 40
nodejs -v20.15.1

[dosubot[bot]]

Hey there, @lalitx17! I'm here to assist you with any questions or issues you have. I can help you solve bugs, answer questions, and become a contributor while you're waiting for a human maintainer. Let me know how I can help!

Yes, there is a way to get the list of collections in langchain/community/vectorstores/chromadb. You can use the index property of the Chroma class, which is an instance of ChromaClientT, to call the listCollections method. Here is an example of how you can do it:

import { Chroma } from "@langchain/community/vectorstores/chroma";
import { HuggingFaceTransformersEmbeddings } from "@langchain/community/embeddings/hf_transformers";

const embeddings = new HuggingFaceTransformersEmbeddings({
  modelName: "Xenova/all-MiniLM-L6-v2",
});

async function listCollections(chromaInstance: Chroma) {
  if (!chromaInstance.index) {
    throw new Error("ChromaClient is not initialized.");
  }
  return await chromaInstance.index.listCollections();
}

async function initChroma() {
  const chromaInstance = await Chroma.fromExistingCollection(embeddings, { collectionName: "your_collection_name" });
  const collections = await listCollections(chromaInstance);
  console.log(collections);
}

initChroma();

This function checks if the index property is initialized and then calls the listCollections method to retrieve the list of collections ^[1].

To continue talking to Dosu, mention @dosu.