query all nodes on a VectorStore based on a metadata filter

[CMauricioAH]

I have on a vector store nodes, that contain as metadata an additional id. I want to filter all of those from the vector store with a query. I found there is the VectorStoreQuery, and added as MetadataFilter the id to search for it, but could just not figure out if this is even possible. So far I receive an empty list as response. Any ideas how to do this?

[dosubot[bot]]

🤖

Hey @CMauricioAH! Great to see you back. Hope you're doing well.

Based on the context provided, it seems like you're on the right track with using the VectorStoreQuery and MetadataFilter. However, it's important to note that not all vector stores in LlamaIndex support metadata filters. The default vector store built into LlamaIndex doesn't support metadata filters. You would have to use something with LlamaIndex that supports it to make your life easier (qdrant, chroma, pinecone, weaviate).

Here is an example of how you can use this feature:

from llama_index import Document
from llama_index.vector_stores.types import ExactMatchFilter, MetadataFilters

# Insert a document with specific metadata
doc = Document(text="target", metadata={"tag": "target"})
index.insert(doc)

# Create a filter that matches the inserted metadata
filters = MetadataFilters(
    filters=[ExactMatchFilter(key="tag", value="target")]
)

# Use the filter in the retriever to retrieve only the documents that match the filter
retriever = index.as_retriever(
    similarity_top_k=20,
    filters=filters,
)

source_nodes = retriever.retrieve("What did the author do growing up?")

This code will retrieve only the documents that have a metadata tag "tag" with the value "target".

For multiple values for a metadata key on the filter, it seems like there is a limitation on the vector store. You might need to find a workaround or use a different vector store that supports this feature.

You can find more information about this in the following issues and pull requests:

• [Question]: Metafilter
• [Feature Request]: Filter on metadata for SimpleVectorstoreindex
• I have converted the document type into a json vector file, if I want to query some questions in multiple json vector files, what should I do?

I hope this helps! If you have any other questions, feel free to ask.

Sources

• [Question]: Filtering on metadata in SimpleVectorStore index
• [Question]: Filtering on metadata in SimpleVectorStore index
• llama-index-legacy/tests/vector_stores/test_rockset.py
• llama-index-core/llama_index/core/vector_stores/simple.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.