Skip to content

Enhanced Metadata Filtering Not Working with Qdrant BackendΒ #3975

@MorseWayne

Description

@MorseWayne

πŸ› Describe the bug

Enhanced Metadata Filtering Not Working with Qdrant Backend

Summary

The enhanced metadata filtering operators documented at https://docs.mem0.ai/open-source/features/metadata-filtering do not work with the Qdrant vector store backend. All operators except simple scalar equality matching fail with ValidationError.

Environment

  • mem0 version: 1.0.3
  • qdrant-client version: 1.16.2
  • Qdrant server version: latest (Docker)
  • Python version: 3.11+
  • OS: Linux

Problem Description

According to the Enhanced Metadata Filtering documentation, Mem0 1.0.0+ should support the following operators:

  • Comparison operators: eq, ne, gt, gte, lt, lte
  • List operators: in, nin
  • String operators: contains, icontains
  • Wildcard: *
  • Logical operators: AND, OR, NOT

The documentation also states that Qdrant supports "Full comparison, list, and logical support. Handles deeply nested boolean logic efficiently."

However, in practice, only simple scalar equality matching works. All other operators fail with Pydantic ValidationError.

Steps to Reproduce

from mem0 import Memory

# Configure with Qdrant backend
config = {
    "vector_store": {
        "provider": "qdrant",
        "config": {
            "host": "localhost",
            "port": 6333,
            "collection_name": "test_collection",
        }
    },
    # ... LLM and embedder config
}

m = Memory.from_config(config)

# Add test memory
m.add(
    messages=[{"role": "user", "content": "Python is my favorite language"}],
    user_id="test_user",
    metadata={"category": "programming", "priority": 10}
)

# This WORKS - simple scalar matching
results = m.search(
    "programming",
    user_id="test_user",
    filters={"category": "programming"}  # βœ… Works
)

# This FAILS - eq operator
results = m.search(
    "programming",
    user_id="test_user",
    filters={"category": {"eq": "programming"}}  # ❌ ValidationError
)

# This FAILS - gte operator
results = m.search(
    "programming",
    user_id="test_user",
    filters={"priority": {"gte": 5}}  # ❌ ValidationError
)

# This FAILS - in operator
results = m.search(
    "programming",
    user_id="test_user",
    filters={"category": {"in": ["programming", "hobby"]}}  # ❌ ValidationError
)

# This FAILS - AND operator
results = m.search(
    "programming",
    user_id="test_user",
    filters={"AND": [{"category": "programming"}, {"priority": 10}]}  # ❌ ValidationError
)

Expected Behavior

All operators documented in the Enhanced Metadata Filtering page should work with Qdrant backend, as stated in the documentation:

Qdrant: Full comparison, list, and logical support. Handles deeply nested boolean logic efficiently.

Actual Behavior

All operators except simple scalar matching fail with:

ValidationError: 3 validation errors for MatchValue
value.bool
  Input should be a valid boolean [type=bool_type, input_value={'eq': 'programming'}, input_type=dict]

Test Results Summary

Operator Type Status Example
Simple scalar match βœ… Works {"category": "programming"}
eq ❌ Fails {"category": {"eq": "programming"}}
ne ❌ Fails {"category": {"ne": "programming"}}
gt ❌ Fails {"priority": {"gt": 7}}
gte ❌ Fails {"priority": {"gte": 8}}
lt ❌ Fails {"priority": {"lt": 8}}
lte ❌ Fails {"priority": {"lte": 7}}
Range (gte+lte) ❌ Fails {"priority": {"gte": 5, "lte": 9}}
in ❌ Fails {"category": {"in": ["a", "b"]}}
nin ❌ Fails {"category": {"nin": ["a"]}}
contains ❌ Fails {"tags": {"contains": "x"}}
icontains ❌ Fails {"tags": {"icontains": "X"}}
AND ❌ Fails {"AND": [{...}, {...}]}
OR ❌ Fails {"OR": [{...}, {...}]}
NOT ❌ Fails {"NOT": [{...}]}

Root Cause Analysis

Looking at the error, it appears the Qdrant vector store implementation (mem0/vector_stores/qdrant.py) constructs filters using MatchValue which only accepts scalar values (bool, int, str), not dict or list structures.

The filter conversion logic doesn't seem to translate the enhanced filter operators (like {"eq": "value"} or {"gte": 5}) into Qdrant's native filter format (which does support these operations via qdrant_client.models.FieldCondition with Match, Range, etc.).

Suggested Fix

The Qdrant vector store implementation should:

  1. Parse the enhanced filter syntax (eq, ne, gt, gte, lt, lte, in, nin, contains, etc.)
  2. Convert them to Qdrant's native filter format using:
    • models.FieldCondition with models.MatchValue for equality
    • models.FieldCondition with models.MatchAny for in operator
    • models.FieldCondition with models.Range for comparison operators
    • models.Filter with must/should/must_not for logical operators

Example of what the conversion should produce for Qdrant:

from qdrant_client import models

# For {"priority": {"gte": 5, "lte": 10}}
qdrant_filter = models.Filter(
    must=[
        models.FieldCondition(
            key="priority",
            range=models.Range(gte=5, lte=10)
        )
    ]
)

# For {"category": {"in": ["programming", "hobby"]}}
qdrant_filter = models.Filter(
    must=[
        models.FieldCondition(
            key="category",
            match=models.MatchAny(any=["programming", "hobby"])
        )
    ]
)

Additional Context

  • Qdrant natively supports all these filter operations - see Qdrant Filtering Documentation
  • The issue is specifically in how mem0 translates the filter syntax to Qdrant's format
  • This significantly limits the usefulness of metadata filtering when using Qdrant backend

Workaround

Currently, users must:

  1. Use only simple scalar matching: {"field": "value"}
  2. Perform multiple queries and merge results in application code for in-like behavior
  3. Implement post-query filtering in application code for complex conditions

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions