Skip to content

Commit 768858e

Browse files
shruti-sridharshruti-sridhar
authored
PYTHON-4630 Add documentation for MongoClient.bulk_write (mongodb#1794)
1 parent ce5c5ad commit 768858e

File tree

9 files changed

+215
-8
lines changed

9 files changed

+215
-8
lines changed

‎.github/workflows/test-python.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ jobs:
9696
- name: Start MongoDB
9797
uses: supercharge/[email protected]
9898
with:
99-
mongodb-version: 4.4
99+
mongodb-version: '8.0.0-rc4'
100100
- name: Run tests
101101
run: |
102102
hatch run doctest:test

‎doc/api/pymongo/asynchronous/mongo_client.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,5 +35,6 @@
3535
.. automethod:: get_database
3636
.. automethod:: server_info
3737
.. automethod:: watch
38+
.. automethod:: bulk_write
3839
.. automethod:: __getitem__
3940
.. automethod:: __getattr__

‎doc/api/pymongo/mongo_client.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,5 +35,6 @@
3535
.. automethod:: get_database
3636
.. automethod:: server_info
3737
.. automethod:: watch
38+
.. automethod:: bulk_write
3839
.. automethod:: __getitem__
3940
.. automethod:: __getattr__

‎doc/changelog.rst

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,19 @@ PyMongo 4.9 brings a number of improvements including:
1616
:class:`~pymongo.asynchronous.cursor.AsyncCursor`,
1717
and :class:`~pymongo.asynchronous.command_cursor.AsyncCommandCursor`
1818
as an asynchronous-friendly alternative to ``list(cursor)``.
19-
19+
- Added :meth:`~pymongo.mongo_client.MongoClient.bulk_write` to :class:`~pymongo.mongo_client.MongoClient`
20+
and :class:`~pymongo.asynchronous.mongo_client.AsyncMongoClient`,
21+
enabling users to perform insert, update, and delete operations
22+
against mixed namespaces in a minimized number of round trips.
23+
Please see :doc:`examples/client_bulk` for more information.
24+
- Added support for the ``namespace`` parameter to the
25+
:class:`~pymongo.operations.InsertOne`,
26+
:class:`~pymongo.operations.ReplaceOne`,
27+
:class:`~pymongo.operations.UpdateOne`,
28+
:class:`~pymongo.operations.UpdateMany`,
29+
:class:`~pymongo.operations.DeleteOne`, and
30+
:class:`~pymongo.operations.DeleteMany` operations, so
31+
they can be used in the new :meth:`~pymongo.mongo_client.MongoClient.bulk_write`.
2032

2133
Issues Resolved
2234
...............

‎doc/examples/client_bulk.rst

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
Client Bulk Write Operations
2+
=============================
3+
4+
.. testsetup::
5+
6+
from pymongo import MongoClient
7+
8+
client = MongoClient()
9+
client.drop_database("client_bulk_example")
10+
db = client.client_bulk_example
11+
client.db.drop_collection("test_one")
12+
client.db.drop_collection("test_two")
13+
client.db.drop_collection("test_three")
14+
client.db.drop_collection("test_four")
15+
client.db.drop_collection("test_five")
16+
client.db.drop_collection("test_six")
17+
18+
The :meth:`~pymongo.mongo_client.MongoClient.bulk_write`
19+
method has been added to :class:`~pymongo.mongo_client.MongoClient` in PyMongo 4.9.
20+
This method enables users to perform batches of write operations **across
21+
multiple namespaces** in a minimized number of round trips, and
22+
to receive detailed results for each operation performed.
23+
24+
.. note:: This method requires MongoDB server version 8.0+.
25+
26+
Basic Usage
27+
------------
28+
29+
A list of insert, update, and delete operations can be passed into the
30+
:meth:`~pymongo.mongo_client.MongoClient.bulk_write` method. Each request
31+
must include the namespace on which to perform the operation.
32+
33+
PyMongo will automatically split the given requests into smaller sub-batches based on
34+
the maximum message size accepted by MongoDB, supporting very large bulk write operations.
35+
36+
The return value is an instance of
37+
:class:`~pymongo.results.ClientBulkWriteResult`.
38+
39+
.. _summary_client_bulk:
40+
41+
Summary Results
42+
.................
43+
44+
By default, the returned :class:`~pymongo.results.ClientBulkWriteResult` instance will contain a
45+
summary of the types of operations performed in the bulk write, along with their respective counts.
46+
47+
.. doctest::
48+
:options: +NORMALIZE_WHITESPACE
49+
50+
>>> from pymongo import InsertOne, DeleteOne, UpdateOne
51+
>>> models = [
52+
... InsertOne(namespace="db.test_one", document={"_id": 1}),
53+
... InsertOne(namespace="db.test_two", document={"_id": 2}),
54+
... DeleteOne(namespace="db.test_one", filter={"_id": 1}),
55+
... UpdateOne(
56+
... namespace="db.test_two",
57+
... filter={"_id": 4},
58+
... update={"$inc": {"j": 1}},
59+
... upsert=True,
60+
... ),
61+
... ]
62+
>>> result = client.bulk_write(models)
63+
>>> result.inserted_count
64+
2
65+
>>> result.deleted_count
66+
1
67+
>>> result.modified_count
68+
0
69+
>>> result.upserted_count
70+
1
71+
72+
.. _verbose_client_bulk:
73+
74+
Verbose Results
75+
.................
76+
77+
If the ``verbose_results`` parameter is set to True, the returned :class:`~pymongo.results.ClientBulkWriteResult`
78+
instance will also include detailed results about each successful operation performed as part of the bulk write.
79+
80+
.. doctest::
81+
:options: +NORMALIZE_WHITESPACE
82+
83+
>>> from pymongo import InsertOne, DeleteMany, ReplaceOne, UpdateMany
84+
>>> models = [
85+
... DeleteMany(
86+
... namespace="db.test_two", filter={}
87+
... ), # Delete all documents from the previous example
88+
... InsertOne(namespace="db.test_one", document={"_id": 1}),
89+
... InsertOne(namespace="db.test_one", document={"_id": 2}),
90+
... InsertOne(namespace="db.test_two", document={"_id": 3}),
91+
... UpdateMany(namespace="db.test_one", filter={}, update={"$set": {"foo": "bar"}}),
92+
... ReplaceOne(
93+
... namespace="db.test_two", filter={"j": 1}, replacement={"_id": 4}, upsert=True
94+
... ),
95+
... ]
96+
>>> result = client.bulk_write(models, verbose_results=True)
97+
>>> result.delete_results
98+
{0: DeleteResult({'ok': 1.0, 'idx': 0, 'n': 2}, ...)}
99+
>>> result.insert_results
100+
{1: InsertOneResult(1, ...),
101+
2: InsertOneResult(2, ...),
102+
3: InsertOneResult(3, ...)}
103+
>>> result.update_results
104+
{4: UpdateResult({'ok': 1.0, 'idx': 4, 'n': 2, 'nModified': 2}, ...),
105+
5: UpdateResult({'ok': 1.0, 'idx': 5, 'n': 1, 'nModified': 0, 'upserted': {'_id': 4}}, ...)}
106+
107+
108+
Handling Errors
109+
----------------
110+
111+
If any errors occur during the bulk write, a :class:`~pymongo.errors.ClientBulkWriteException` will be raised.
112+
If a server, connection, or network error occurred, the ``error`` field of the exception will contain
113+
that error.
114+
115+
Individual write errors or write concern errors get recorded in the ``write_errors`` and ``write_concern_errors`` fields of the exception.
116+
The ``partial_result`` field gets populated with the results of any operations that were successfully completed before the exception was raised.
117+
118+
.. _ordered_client_bulk:
119+
120+
Ordered Operations
121+
....................
122+
123+
In an ordered bulk write (the default), if an individual write fails, no further operations will get executed.
124+
For example, a duplicate key error on the third operation below aborts the remaining two operations.
125+
126+
.. doctest::
127+
:options: +NORMALIZE_WHITESPACE
128+
129+
>>> from pymongo import InsertOne, DeleteOne
130+
>>> from pymongo.errors import ClientBulkWriteException
131+
>>> models = [
132+
... InsertOne(namespace="db.test_three", document={"_id": 3}),
133+
... InsertOne(namespace="db.test_four", document={"_id": 4}),
134+
... InsertOne(namespace="db.test_three", document={"_id": 3}), # Duplicate _id
135+
... InsertOne(namespace="db.test_four", document={"_id": 5}),
136+
... DeleteOne(namespace="db.test_three", filter={"_id": 3}),
137+
... ]
138+
>>> try:
139+
... client.bulk_write(models)
140+
... except ClientBulkWriteException as cbwe:
141+
... exception = cbwe
142+
...
143+
>>> exception.write_errors
144+
[{'ok': 0.0,
145+
'idx': 2,
146+
'code': 11000,
147+
'errmsg': 'E11000 duplicate key error ... dup key: { _id: 3 }', ...
148+
'op': {'insert': 'db.test_three', 'document': {'_id': 3}}}]
149+
>>> exception.partial_result.inserted_count
150+
2
151+
>>> exception.partial_result.deleted_count
152+
0
153+
154+
.. _unordered_client_bulk:
155+
156+
Unordered Operations
157+
.....................
158+
159+
If the ``ordered`` parameter is set to False, all operations in the bulk write will be attempted, regardless of any individual write errors that occur.
160+
For example, the fourth and fifth write operations below get executed successfully, despite the duplicate key error on the third operation.
161+
162+
.. doctest::
163+
:options: +NORMALIZE_WHITESPACE
164+
165+
>>> from pymongo import InsertOne, DeleteOne
166+
>>> from pymongo.errors import ClientBulkWriteException
167+
>>> models = [
168+
... InsertOne(namespace="db.test_five", document={"_id": 5}),
169+
... InsertOne(namespace="db.test_six", document={"_id": 6}),
170+
... InsertOne(namespace="db.test_five", document={"_id": 5}), # Duplicate _id
171+
... InsertOne(namespace="db.test_six", document={"_id": 7}),
172+
... DeleteOne(namespace="db.test_five", filter={"_id": 5}),
173+
... ]
174+
>>> try:
175+
... client.bulk_write(models, ordered=False)
176+
... except ClientBulkWriteException as cbwe:
177+
... exception = cbwe
178+
...
179+
>>> exception.write_errors
180+
[{'ok': 0.0,
181+
'idx': 2,
182+
'code': 11000,
183+
'errmsg': 'E11000 duplicate key error ... dup key: { _id: 5 }', ...
184+
'op': {'insert': 'db.test_five', 'document': {'_id': 5}}}]
185+
>>> exception.partial_result.inserted_count
186+
3
187+
>>> exception.partial_result.deleted_count
188+
1

‎doc/examples/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ MongoDB, you can start it like so:
2222
copydb
2323
custom_type
2424
bulk
25+
client_bulk
2526
datetimes
2627
geo
2728
gevent

‎pymongo/asynchronous/mongo_client.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2274,8 +2274,8 @@ async def bulk_write(
22742274
1
22752275
>>> result.modified_count
22762276
0
2277-
>>> result.upserted_ids
2278-
{3: ObjectId('54f62ee28891e756a6e1abd5')}
2277+
>>> result.upserted_count
2278+
1
22792279
>>> async for doc in db.test.find({}):
22802280
... print(doc)
22812281
...
@@ -2312,6 +2312,8 @@ async def bulk_write(
23122312
23132313
:return: An instance of :class:`~pymongo.results.ClientBulkWriteResult`.
23142314
2315+
.. seealso:: For more info, see :doc:`/examples/client_bulk`.
2316+
23152317
.. seealso:: :ref:`writes-and-ids`
23162318
23172319
.. note:: requires MongoDB server version 8.0+.

‎pymongo/errors.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -342,10 +342,10 @@ def write_concern_errors(self) -> Optional[list[WriteConcernError]]:
342342
return self.details.get("writeConcernErrors", [])
343343

344344
@property
345-
def write_errors(self) -> Optional[Mapping[int, WriteError]]:
345+
def write_errors(self) -> Optional[list[WriteError]]:
346346
"""Errors that occurred during the execution of individual write operations.
347347
348-
This map will contain at most one entry if the bulk write was ordered.
348+
This list will contain at most one entry if the bulk write was ordered.
349349
"""
350350
return self.details.get("writeErrors", {})
351351

‎pymongo/synchronous/mongo_client.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2263,8 +2263,8 @@ def bulk_write(
22632263
1
22642264
>>> result.modified_count
22652265
0
2266-
>>> result.upserted_ids
2267-
{3: ObjectId('54f62ee28891e756a6e1abd5')}
2266+
>>> result.upserted_count
2267+
1
22682268
>>> for doc in db.test.find({}):
22692269
... print(doc)
22702270
...
@@ -2301,6 +2301,8 @@ def bulk_write(
23012301
23022302
:return: An instance of :class:`~pymongo.results.ClientBulkWriteResult`.
23032303
2304+
.. seealso:: For more info, see :doc:`/examples/client_bulk`.
2305+
23042306
.. seealso:: :ref:`writes-and-ids`
23052307
23062308
.. note:: requires MongoDB server version 8.0+.

0 commit comments

Comments
 (0)