Meta Data filtering for Amazon document db

[kartiksonaghela]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

agronomy_collection = DocumentDBVectorSearch.from_connection_string(
connection_string=connection_string,
namespace = agronomy_name_space, 
index_name = "vector",
embedding = embeddings
)

# Create retriever with filtering
agronomy_retriever = agronomy_collection.as_retriever(
    search_type="similarity",
    #search_kwargs={"k": 10},
    search_kwargs={'filter': {'crop name':'corn'}}
    
)

# Perform semantic matching with filter applied
query = 'best farming method'
semantic_matches = agronomy_retriever.invoke(query)
##This index for semantic search
collection.create_index ([("vectorContent","vector")], 
    vectorOptions= {
        "type": "hnsw", 
        "similarity": "cosine",
        "dimensions": 1536,
        "m": 16,
        "efConstruction": 64},
    name="vector")
## This is how i have created a text search index
collection.create_index({"crop name": "text", "product name":"text","seed product name":"text","product category":"text"},name="text_index")

Description

Hii, I am creating a RAG application for which i want better retrieval results so for that i want to use metadata filtering feature where we filter based on provide metadata and then carry out semantic search on the filtered result. I tried but it is not working
Is this feature which is not available in Amazon document db?

System Info

langchain==0.1.14
langchain_community==0.0.31
langchain_openai==0.1.1
langcodes==3.3.0
openai python-dotenv==1.0.1
language_data==1.1
pymongo==4.7.3

[feijoes]

The search_kwargs={'filter': {'crop name':'corn'}} filter should be working, check if the key and value are correct

[kartiksonaghela]

@feijoes thanks for the reply but i have check the key and value are correct

this are my indexes which i have created for text search which are the metadata for my chunks so key value i don't see there is a problem.

[kartiksonaghela]

Also there is an additional reference for my chunk metadata

[feijoes]

@kartiksonaghela try filtering by another metadata like default_language and see if that works

[kartiksonaghela]

@feijoes i tried with other metadata like product category, product name, seed product name. It is not working for any of them
agronomy_name_space='test.agronomy_product'
client = MongoClient(connection_string)
collection = client['test']['agronomy_product']
#collection=client['chatbot-service-qa']['agronomy_product']
agronomy_collection = DocumentDBVectorSearch.from_connection_string(
connection_string=connection_string,
namespace = agronomy_name_space,
index_name = "vector",
embedding = embeddings
)

Create retriever with filtering

agronomy_retriever = agronomy_collection.as_retriever(
search_type="similarity",
#search_kwargs={"k": 10},
search_kwargs={'filter': {'product category': 'Fungicide'}},

)

Perform semantic matching with filter applied

query = 'how to use this for farming'
semantic_matches = agronomy_retriever.invoke(query)