Skip to content

Conversation

Jibola
Copy link
Contributor

@Jibola Jibola commented Aug 21, 2025

Summary

This PR introduces an abstraction to convert some of our simple $expr queries to $match queries without $expr in them. If the $expr query cannot be converted to a $match query, it will leave it as a $expr. This behavior is nested. Currently this is limited to $eq and $in operations for now, but in the future can be extended to other $expr conversions.

This only affects the querying done in self.match_mql as the querying for lookups, groupings, or aggregation pipelines are out of scope for this pr.

Changes in this PR

  • Introduce the QueryOptimizer abstraction (which can be changed; it doesn't store state so it does not need to remain an object)
  • Introduced ExpressionConverters which house the logic for converting queries from $expr usage to their $match equivalents.
  • Added several tests to confirm the behavior of the $eq, $in, $and, $or in drilling down of expression conversion.

Test Plan

  • Manual testing through python manage.py shell See an example query below.
  • Added test cases. Some old tests will fail and need to be fixed

Screenshots (grabbed text)

# Before
>>> Author.objects.filter(author_city__in=["London"])
{
    "$match": {
        "$expr": {
            "$and": [
                "$in": ["$author_city", "London"]
            ]
        }
    }
}

# After 
>>> Author.objects.filter(author_city__in=["London"])
{
    "$match": {
        "$and": {
            "$in": ["$author_city", "London"]
        }
    }
}

Callouts

  • This is still a proof of concept. Like said above, QueryOptimizer should be removed and left as its functions if it is not going to retain state. Initialization is unnecessary.
  • This will not work on Embedded Models until a$getField converter is made.
  • This PR only assumes self.match_mql resolves to a $expr (which is true) future work should include keeping any query that doesn't use self.match_mql.

@Jibola Jibola changed the title PoC Expression Conversion Abstraction [INTPYTHON-TODO]: PoC Expression Conversion Abstraction Aug 25, 2025
@NoahStapp NoahStapp changed the title [INTPYTHON-TODO]: PoC Expression Conversion Abstraction INTPYTHON-736 - Convert simple $expr queries to $match queries Aug 27, 2025
@NoahStapp
Copy link

The remaining test failure test_changelist_view_list_editable_changed_objects_uses_filter will be fixed by an upcoming PR to our django fork.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants