how to get relevant documents using Chroma while using ParentDocumentRetriever?

[nithinreddyyyyyy]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

child_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
parent_splitter = RecursiveCharacterTextSplitter(chunk_size=1200, chunk_overlap=300)

vectorstore = Chroma(
  collection_name="full_documents", embedding_function=embeddings)

store = InMemoryStore()

retriever = ParentDocumentRetriever(
  vectorstore=vectorstore,
  docstore=store,
  child_splitter=child_splitter,
  parent_splitter=parent_splitter
)

retriever.add_documents(documents, ids=None)

Description

I've given the code above, i'm unable to find any code like how to retrieve the documents using the above retriever, i tried the below code, but it is not working

parent_document_retriever.get_relevant_documents(query)

and also i need to set the topk limit. Can you help me with the code?

System Info

google colab

[dosubot[bot]]

🤖

Hey @nithinreddyyyyyy! Great to see you back and diving into more complex features of LangChain. How's everything going on your end?

Based on the code you've provided, it seems like you're trying to retrieve relevant documents using the ParentDocumentRetriever class in the LangChain framework. However, the ParentDocumentRetriever class doesn't have a get_relevant_documents method. Instead, it has an add_documents method which is used to add documents to the docstore and vectorstores.

The get_relevant_documents method is actually part of the BaseRetriever class in LangChain, which is used to retrieve documents relevant to a given query. It's an abstract method that should be implemented by subclasses of BaseRetriever.

If you want to retrieve relevant documents, you might want to use a different retriever that implements this method. For example, the SelfQueryRetriever class has a get_relevant_documents method that you can use. Here's an example of how you can use it:

# Instantiate the SelfQueryRetriever with enable_limit=True
retriever = SelfQueryRetriever.from_llm(
    llm,
    vectorstore,
    document_content_description,
    metadata_field_info,
    enable_limit=True,
    verbose=True,
)

# Retrieve two relevant documents about dinosaurs
retriever.get_relevant_documents("what are two movies about dinosaurs")

In this example, the get_relevant_documents method is called with the query "what are two movies about dinosaurs". The enable_limit=True argument in the SelfQueryRetriever constructor allows the retriever to limit the number of documents returned based on the number specified in the query.

You can find more information about this in the Chroma Self Query notebook in the LangChain documentation.

If you're specifically interested in using the ParentDocumentRetriever class, you might want to look into how it works. The ParentDocumentRetriever class is designed to handle the trade-off between having small documents for accurate embeddings and having long enough documents to retain context. It works by splitting and storing small chunks of data. During retrieval, it first fetches the small chunks but then looks up the parent ids for those chunks and returns those larger documents. You can find more information about this in the ParentDocumentRetriever source code.

I hope this helps! If you have any other questions, feel free to ask.

Sources

• docs/docs/integrations/retrievers/self_query/chroma_self_query.ipynb
• libs/langchain/langchain/retrievers/parent_document_retriever.py
• libs/langchain/langchain/schema/retriever.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Zalasyu]

Would you say the Self-querying strategy which helps for query construction is like Hybrid Search? Also I would look into different more advanced text splitting strategies. The recursive splitting is a great start but depending on the kind of data ou are working with the chunking strategy may break similar sentences into different chunks instead of keeping it together as one.