Skip to content

Commit 5532c6a

Browse files
committed
<ADD>: async_insert
1 parent 615e453 commit 5532c6a

File tree

5 files changed

+708
-0
lines changed

5 files changed

+708
-0
lines changed

ASYNC_AGGREGATE_FIX.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Async Aggregate Fix Documentation
2+
3+
## Problem
4+
`async_aggregate()` was returning a coroutine instead of an async iterator, causing `TypeError: 'async for' requires an object with __aiter__ method, got coroutine` when trying to use it directly with `async for`.
5+
6+
## Solution
7+
Implemented `AsyncAggregationIterator` wrapper class that provides the async iterator interface, allowing direct use with `async for`.
8+
9+
### Implementation Details
10+
11+
1. **New Class**: `AsyncAggregationIterator` in `mongoengine/async_iterators.py`
12+
- Implements `__aiter__` and `__anext__` methods
13+
- Lazy execution - aggregation pipeline runs on first iteration
14+
- Handles all QuerySet options (filters, skip, limit, ordering)
15+
16+
2. **Modified Method**: `async_aggregate()` in `BaseQuerySet`
17+
- Changed from async function to regular function
18+
- Returns `AsyncAggregationIterator` instance instead of coroutine
19+
- No breaking changes for the API
20+
21+
### Usage
22+
23+
**Before (would cause error):**
24+
```python
25+
# This would fail with TypeError
26+
async for poi in Model.objects.async_aggregate(pipeline):
27+
process(poi)
28+
```
29+
30+
**After (works correctly):**
31+
```python
32+
# Now works directly with async for
33+
async for poi in Model.objects.async_aggregate(pipeline):
34+
process(poi)
35+
```
36+
37+
### Examples
38+
39+
```python
40+
# Basic aggregation
41+
pipeline = [
42+
{"$match": {"status": "active"}},
43+
{"$group": {"_id": "$category", "count": {"$sum": 1}}}
44+
]
45+
46+
async for result in TravelPoiBounds.objects.async_aggregate(pipeline):
47+
print(f"Category: {result['_id']}, Count: {result['count']}")
48+
49+
# With QuerySet operations
50+
async for result in (
51+
Model.objects
52+
.filter(active=True)
53+
.order_by("-created")
54+
.limit(10)
55+
.async_aggregate(pipeline)
56+
):
57+
process(result)
58+
59+
# Complex pipeline with $lookup
60+
pipeline = [
61+
{
62+
"$lookup": {
63+
"from": "authors",
64+
"localField": "author_id",
65+
"foreignField": "_id",
66+
"as": "author_details"
67+
}
68+
},
69+
{"$unwind": "$author_details"},
70+
{"$project": {
71+
"title": 1,
72+
"author_name": "$author_details.name"
73+
}}
74+
]
75+
76+
async for book in Book.objects.async_aggregate(pipeline):
77+
print(f"{book['title']} by {book['author_name']}")
78+
```
79+
80+
### Benefits
81+
82+
1. **Intuitive API**: Works like other async iterables in Python
83+
2. **Lazy Execution**: Pipeline only executes when iteration starts
84+
3. **Memory Efficient**: Results are streamed, not loaded all at once
85+
4. **Backward Compatible**: Existing code structure maintained
86+
87+
### Migration
88+
89+
No migration needed - the new implementation works with the natural async for pattern that users expect.

ASYNC_SELECT_RELATED_FIX.md

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# Async Select Related Fix Documentation
2+
3+
## Problem
4+
When using `select_related()` with async QuerySets in MongoEngine, a `TypeError: 'AsyncCursor' object is not an iterator` error occurs because the synchronous `select_related()` method tries to iterate over an `AsyncCursor` using synchronous iteration.
5+
6+
## Solution
7+
We've implemented a new `async_select_related()` method that properly handles async cursors and dereferences.
8+
9+
### Implementation Details
10+
11+
1. **New Method**: `async_select_related(max_depth=1)` in `BaseQuerySet`
12+
2. **New Class**: `AsyncDeReference` in `mongoengine/dereference.py` that handles async operations
13+
3. **New Method**: `async_in_bulk(object_ids)` for efficient bulk fetching in async context
14+
15+
### Usage
16+
17+
Instead of:
18+
```python
19+
# This will fail with TypeError
20+
items = await self.skip(skip).limit(limit + 1).select_related().async_to_list()
21+
```
22+
23+
Use:
24+
```python
25+
# This works correctly
26+
items = await self.skip(skip).limit(limit + 1).async_select_related()
27+
```
28+
29+
### Example
30+
31+
```python
32+
from mongoengine import Document, ReferenceField, StringField, connect_async
33+
34+
class Author(Document):
35+
name = StringField(required=True)
36+
37+
class Post(Document):
38+
title = StringField(required=True)
39+
author = ReferenceField(Author)
40+
41+
class Comment(Document):
42+
content = StringField(required=True)
43+
post = ReferenceField(Post)
44+
author = ReferenceField(Author)
45+
46+
# Connect to async MongoDB
47+
await connect_async('mydb')
48+
49+
# Create data
50+
author = Author(name="John Doe")
51+
await author.async_save()
52+
53+
post = Post(title="Hello World", author=author)
54+
await post.async_save()
55+
56+
# Query with select_related
57+
posts = await Post.objects.async_select_related()
58+
# posts[0].author is now the actual Author document, not a reference
59+
60+
# With nested references (max_depth=2)
61+
comments = await Comment.objects.async_select_related(max_depth=2)
62+
# comments[0].post and comments[0].author are dereferenced
63+
# Note: comments[0].post.author remains an AsyncReferenceProxy
64+
```
65+
66+
### Limitations
67+
68+
1. **Nested References**: With `max_depth > 1`, deeply nested references may still be `AsyncReferenceProxy` objects that require explicit fetching:
69+
```python
70+
# For deeply nested references
71+
author = await comment.post.author.fetch()
72+
```
73+
74+
2. **Performance**: Like the sync version, `async_select_related()` performs additional queries to fetch all referenced documents. Use it when you know you'll need the referenced data.
75+
76+
### Migration from Sync to Async
77+
78+
If you're migrating from synchronous to asynchronous MongoEngine:
79+
80+
1. Replace `select_related()` with `async_select_related()`
81+
2. Replace `in_bulk()` with `async_in_bulk()` if used directly
82+
3. Be aware that async operations return `AsyncReferenceProxy` for references by default
83+
84+
### Testing
85+
86+
The implementation includes comprehensive tests in `tests/test_async_select_related.py` covering:
87+
- Single reference dereferencing
88+
- Multiple reference fields
89+
- Filtering with select_related
90+
- Skip/limit with select_related
91+
- Empty querysets
92+
- Max depth handling

0 commit comments

Comments
 (0)