diff --git a/source/aggregation/aggregation-tutorials/filtered-subset.txt b/source/aggregation/aggregation-tutorials/filtered-subset.txt index 8bf3c50c..e9a738b1 100644 --- a/source/aggregation/aggregation-tutorials/filtered-subset.txt +++ b/source/aggregation/aggregation-tutorials/filtered-subset.txt @@ -90,9 +90,7 @@ Tutorial Next, add a :manual:`$sort ` stage that sorts the documents in descending order by the ``dateofbirth`` field to - list the youngest people first. Because Python dictionaries don't maintain the - order of their elements, use a ``SON``or ``OrderedDict`` object - instead: + list the youngest people first: .. literalinclude:: /includes/aggregation/filtered-subset.py :language: python diff --git a/source/connect/mongoclient.txt b/source/connect/mongoclient.txt index 82d4f671..c2b31266 100644 --- a/source/connect/mongoclient.txt +++ b/source/connect/mongoclient.txt @@ -119,7 +119,7 @@ constructor accepts. All parameters are optional. :wikipedia:`round-robin DNS ` addresses. **Data type:** ``Union[str, Sequence[str]]`` - **Default value:** ``"localhost"`` + | **Default value:** ``"localhost"`` * - ``port`` - The port number {+mdb-server+} is running on. @@ -128,17 +128,35 @@ constructor accepts. All parameters are optional. instead of using this parameter. **Data type:** ``int`` - **Default value:** ``27017`` + | **Default value:** ``27017`` * - ``document_class`` - The default class that the client uses to decode BSON documents returned by queries. - This parameter supports the ``bson.raw_bson.RawBSONDocument`` type, as well as - subclasses of the ``collections.abc.Mapping`` type, such as ``bson.son.SON``. + This parameter accepts the following types: - If you specify ``bson.son.SON`` as the document class, you must also specify types - for the key and value. + - ``bson.raw_bson.RawBSONDocument``. To learn more about the ``RawBSONDocument`` class, + see :ref:`pymongo-bson-raw`. + + - A subclass of the ``collections.abc.Mapping`` type, such as ``OrderedDict``. + Depending on the strictness of your type-checking rules, you might also need to + specify types for the key and value, as shown in the following example: + + .. code-block:: python + + client = MongoClient(document_class=OrderedDict[str, int]) + + - A subclass of the ``TypedDict`` type. To pass a ``TypedDict`` subclass for this + parameter, you must also include the class in a type hint for your ``MongoClient`` + object, as shown in the following example: + + .. code-block:: python + + client: MongoClient[MyTypedDict] = MongoClient() + + .. include:: /includes/type-hints/typeddict-availability.rst **Data type:** ``Type[_DocumentType]`` + **Default:** ``dict`` * - ``tz_aware`` - If this parameter is ``True``, the client treats ``datetime`` values as aware. @@ -226,28 +244,46 @@ To use multiprocessing with {+driver-short+}, write code similar to the followin Do not copy an instance of the ``MongoClient`` class from the parent process to a child process. -Type Hints +.. _pymongo-type-hints-client: + +Type Hints ---------- -If you're using Python v3.5 or later, you can add type hints to your Python code. +.. include:: /includes/type-hints/intro.rst -The following code example shows how to declare a type hint for a ``MongoClient`` -object: +To use type hints in your {+driver-short+} application, you must add a type annotation to your +``MongoClient`` object, as shown in the following example: .. code-block:: python client: MongoClient = MongoClient() -In the previous example, the code doesn't specify a type for the documents that the -``MongoClient`` object will work with. To specify a document type, -include the ``Dict[str, Any]`` type when you -create the ``MongoClient`` object, as shown in the following example: +For more accurate type information, you can include the generic document type +``Dict[str, Any]`` in your type annotation. This data type matches all documents in MongoDB. +The following example shows how to include this data type in your type annotation: .. code-block:: python from typing import Any, Dict client: MongoClient[Dict[str, Any]] = MongoClient() +If all the documents that you are working with correspond to a single custom type, you +can specify the custom type as a type hint for your ``MongoClient`` object. This +provides more accurate type information than the generic ``Dict[str, Any]`` type. + +The following example shows how to specify the ``Movie`` type as a type hint for a +``MongoClient`` object: + +.. code-block:: python + + from typing import TypedDict + + class Movie(TypedDict): + name: str + year: int + + client: MongoClient[Movie] = MongoClient() + Troubleshooting --------------- @@ -312,10 +348,14 @@ process. multithreaded contexts with ``fork()``, see `Issue 6721 `__ in the Python Issue Tracker. +.. include:: /includes/type-hints/troubleshooting-client-type.rst + +.. include:: /includes/type-hints/troubleshooting-incompatible-type.rst + API Documentation ----------------- To learn more about creating a ``MongoClient`` object in {+driver-short+}, see the following API documentation: -- `MongoClient <{+api-root+}pymongo/mongo_client.html#pymongo.mongo_client.MongoClient>`__ \ No newline at end of file +- `MongoClient <{+api-root+}pymongo/mongo_client.html#pymongo.mongo_client.MongoClient>`__ \ No newline at end of file diff --git a/source/data-formats/bson.txt b/source/data-formats/bson.txt index 3a91f882..79d39037 100644 --- a/source/data-formats/bson.txt +++ b/source/data-formats/bson.txt @@ -117,6 +117,8 @@ The following example reads the sample BSON document from ``file.bson``: {"address": {"street": "Pizza St", "zipcode": "10003"}, "coord": [-73.982419, 41.579505], "cuisine": "Pizza", "name": "Mongo's Pizza"} +.. _pymongo-bson-raw: + Work with Raw BSON Data ----------------------- diff --git a/source/databases-collections.txt b/source/databases-collections.txt index 0b7ee96d..f50ca2cf 100644 --- a/source/databases-collections.txt +++ b/source/databases-collections.txt @@ -321,9 +321,79 @@ To learn more about supported retryable read operations, see :manual:`Retryable in the {+mdb-server+} manual. To learn more about supported retryable write operations, see :manual:`Retryable Writes ` in the {+mdb-server+} manual. +.. _pymongo-databases-collection-type-hints: + +Type Hints +---------- + +.. include:: /includes/type-hints/intro.rst + +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst + +Database +~~~~~~~~ + +If all documents in a database match a well-defined schema, you can specify a type hint +that uses a Python class to represent the documents' structure. By including this class +in the type hint for your ``Database`` object, you can ensure that all documents you +store or retrieve have the required structure. This provides more accurate type +checking and code completion than the default ``Dict[str, Any]`` type. + +First, define a class to represent a document from the database. The class must inherit +from the ``TypedDict`` class and must contain the same fields as the documents in the +database. After you define your class, include its name as the generic type for the +``Database`` type hint. + +The following example defines a ``Movie`` class and uses it as the +generic type for a ``Database`` type hint: + +.. code-block:: python + :emphasize-lines: 5-7, 10 + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.database import Database + + class Movie(TypedDict): + name: str + year: int + + client: MongoClient = MongoClient() + database: Database[Movie] = client["test_database"] + +Collection +~~~~~~~~~~ + +Adding a generic type to a ``Collection`` type hint is similar to adding a generic type +to a ``Database`` type hint. First, define a class that inherits from the ``TypedDict`` class +and represents the structure of the +documents in the collection. Then, include the class name as the generic type for the +``Collection`` type hint, as shown in the following example: + +.. code-block:: python + :emphasize-lines: 5-7,11 + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.collection import Collection + + class Movie(TypedDict): + name: str + year: int + + client: MongoClient = MongoClient() + database = client["test_database"] + collection: Collection[Movie] = database["test_collection"] + Troubleshooting --------------- +.. include:: /includes/type-hints/troubleshooting-client-type.rst + +.. include:: /includes/type-hints/troubleshooting-incompatible-type.rst + .. include:: /includes/troubleshooting/read-write-options.rst API Documentation diff --git a/source/fundamentals/type-hints.txt b/source/fundamentals/type-hints.txt deleted file mode 100644 index cb8114c8..00000000 --- a/source/fundamentals/type-hints.txt +++ /dev/null @@ -1,435 +0,0 @@ -.. _pymongo-type-hints: - -Type Hints -========== - -.. contents:: On this page - :local: - :backlinks: none - :depth: 2 - :class: singlecol - -.. facet:: - :name: genre - :values: tutorial - -.. meta:: - :keywords: mypy, type safety, compile time, type check, static - -If your IDE is configured to use `type hints `__, -it can suggest more appropriate completions and highlight errors in your code. - -All of the public APIs in PyMongo are fully type hinted, and several of them support -generic parameters for the type of document object returned when decoding BSON documents. - -For a larger set of examples that use types, see the PyMongo -`test_typing module `__. - -.. note:: - - You can also use the `Mypy `__ - tool from your command line or in continuous-integration tests. However, due to - `limitations in Mypy `__, the default - values for generic document types are not yet available. They will eventually be - ``Dict[str, any]``). - - If you are using Mypy and would like to opt out of using the provided types, add the - following lines to your - `Mypy configuration file `__: - - .. code-block:: python - - [mypy-pymongo] - follow_imports = False - -Basic Usage ------------ - -The following code example specifies the ``MongoClient`` type for the ``MongoClient`` object, but -doesn't specify a type for documents. Therefore, {+driver-short+} -uses the default, unspecified type for these documents: - -.. code-block:: python - :emphasize-lines: 2 - - >>> from pymongo import MongoClient - >>> client: MongoClient = MongoClient() - >>> collection = client.test.test - >>> inserted = collection.insert_one({"x": 1, "tags": ["dog", "cat"]}) - >>> retrieved = collection.find_one({"x": 1}) - >>> assert isinstance(retrieved, dict) - -To specify a type for documents, you can use the ``Dict[str, Any]`` type when you -create the ``MongoClient`` object, as shown in the following example: - -.. code-block:: python - :emphasize-lines: 1,3 - - >>> from typing import Any, Dict - >>> from pymongo import MongoClient - >>> client: MongoClient[Dict[str, Any]] = MongoClient() - >>> collection = client.test.test - >>> inserted = collection.insert_one({"x": 1, "tags": ["dog", "cat"]}) - >>> retrieved = collection.find_one({"x": 1}) - >>> assert isinstance(retrieved, dict) - -Typed Client ------------- - -When you create a ``MongoClient`` object, you can specify the document type used to -decode BSON documents. - -The following example shows how to specify a ``~bson.raw_bson.RawBSONDocument`` document type: - -.. code-block:: python - - >>> from pymongo import MongoClient - >>> from bson.raw_bson import RawBSONDocument - >>> client = MongoClient(document_class=RawBSONDocument) - >>> collection = client.test.test - >>> inserted = collection.insert_one({"x": 1, "tags": ["dog", "cat"]}) - >>> result = collection.find_one({"x": 1}) - >>> assert isinstance(result, RawBSONDocument) - -You can also use subclasses of ``collections.abc.Mapping`` such as ``~bson.son.SON``, -as shown in the following example: - -.. code-block:: python - - >>> from bson import SON - >>> from pymongo import MongoClient - >>> client = MongoClient(document_class=SON[str, int]) - >>> collection = client.test.test - >>> inserted = collection.insert_one({"x": 1, "y": 2}) - >>> result = collection.find_one({"x": 1}) - >>> assert result is not None - >>> assert result["x"] == 1 - -.. note:: - - When you use the ``~bson.son.SON`` document type, you must also specify types - for the key and value. The preceding example uses ``SON[str, int]``. - -Typed Collection ----------------- - -If you use a well-defined schema for the the data in a -``~pymongo.collection.Collection``, you can use the ``~typing.TypedDict`` class -to declare types for the values of the elements in the collection. - -.. important:: - - The ``TypedDict`` class is available only in Python 3.8 and later. - To use ``TypedDict`` in earlier versions of Python, install the ``typing_extensions`` - package. - -In the following example, ``Movie`` is an instance of ``TypedDict``. Each ``Movie`` object -contains two key-value pairs: ``name``, a string key with a string value, and -``year``, a string key with an integer value. - -.. code-block:: python - :emphasize-lines: 1, 4-6 - - >>> from typing import TypedDict - >>> from pymongo import MongoClient - >>> from pymongo.collection import Collection - >>> class Movie(TypedDict): - ... name: str - ... year: int - ... - >>> client: MongoClient = MongoClient() - >>> collection: Collection[Movie] = client.test.test - >>> inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> assert result["year"] == 1993 - -This same typing scheme works for all of the insert methods: - -- ``~pymongo.collection.Collection.insert_one()`` -- ``~pymongo.collection.Collection.insert_many()`` -- ``~pymongo.collection.Collection.bulk_write()`` - -For ``bulk_write()``, both the ``~pymongo.operations.InsertOne()`` and -``~pymongo.operations.ReplaceOne()`` operators are generic. - -The following code example shows that the results are the same as the preceding examples -when you call the ``bulk_write()`` method: - -.. code-block:: python - - >>> from typing import TypedDict - >>> from pymongo import MongoClient - >>> from pymongo.operations import InsertOne - >>> from pymongo.collection import Collection - >>> client: MongoClient = MongoClient() - >>> collection: Collection[Movie] = client.test.test - >>> inserted = collection.bulk_write([InsertOne(Movie(name="Jurassic Park", year=1993))]) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> assert result["year"] == 1993 - >>> # This will raise a type-checking error, despite being present, because it is added by PyMongo. - >>> assert result["_id"] # type:ignore[typeddict-item] - -Modeling Document Types with TypedDict --------------------------------------- - -All `schema validation `__ -for inserts and updates is done on the server. These methods automatically add an ``_id`` -field to each document that doesn't include one. There are three ways to handle the ``_id`` -field in your custom ``TypedDict`` class. - -If you don't specify the ``_id`` field, {+driver-short+} automatically inserts it. -You can retrieve the value of the field at runtime, but you'll see a type error -at compile time, as shown in the following example: - -.. code-block:: python - :emphasize-lines: 12-13 - - >>> from typing import TypedDict - >>> from pymongo import MongoClient - >>> from pymongo.collection import Collection - >>> class Movie(TypedDict): - ... name: str - ... year: int - ... - >>> client: MongoClient = MongoClient() - >>> collection: Collection[Movie] = client.test.test - >>> inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> # _id is present but was added by PyMongo; this will raise a type-checking error - >>> assert result["_id"] - -You can ignore this type error by adding a ``# type:ignore`` comment at the end of -the line, as shown in the following example: - -.. code-block:: python - :emphasize-lines: 13 - - >>> from typing import TypedDict - >>> from pymongo import MongoClient - >>> from pymongo.collection import Collection - >>> class Movie(TypedDict): - ... name: str - ... year: int - ... - >>> client: MongoClient = MongoClient() - >>> collection: Collection[Movie] = client.test.test - >>> inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> # _id is present but was added by PyMongo; this will raise a type-checking error - >>> assert result["_id"] # type:ignore[typeddict-item] - -If you explicitly specify a value for the ``_id`` field, every instance of your custom -``TypedDict`` class must have a value for ``_id``. - -A third alternative is to install the ``~typing.NotRequired`` package. When you use this -package, you can access the ``_id`` field at run-time, but you won't see a type error, and -you don't need to include the field in every instance of your class. - -The following example shows how to implement these three approaches to the ``_id`` field: - -.. code-block:: python - - >>> from typing import TypedDict, NotRequired - >>> from pymongo import MongoClient - >>> from pymongo.collection import Collection - >>> from bson import ObjectId - >>> class Movie(TypedDict): - ... name: str - ... year: int - ... - >>> class ExplicitMovie(TypedDict): - ... _id: ObjectId - ... name: str - ... year: int - ... - >>> class NotRequiredMovie(TypedDict): - ... _id: NotRequired[ObjectId] - ... name: str - ... year: int - ... - >>> client: MongoClient = MongoClient() - >>> collection: Collection[Movie] = client.test.test - >>> inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> # This will yield a type-checking error, despite being present, because it is added by PyMongo. - >>> assert result["_id"] # type:ignore[typeddict-item] - >>> collection: Collection[ExplicitMovie] = client.test.test - >>> # Note that the _id keyword argument must be supplied - >>> inserted = collection.insert_one( - ... ExplicitMovie(_id=ObjectId(), name="Jurassic Park", year=1993) - ... ) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> # This will not raise a type-checking error. - >>> assert result["_id"] - >>> collection: Collection[NotRequiredMovie] = client.test.test - >>> # Note the lack of _id, similar to the first example - >>> inserted = collection.insert_one(NotRequiredMovie(name="Jurassic Park", year=1993)) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> # This will not raise a type-checking error, despite not being provided explicitly. - >>> assert result["_id"] - -.. important:: - - The ``NotRequired`` class is available only in Python 3.11 and later. - To use ``NotRequired`` in earlier versions of Python, install the ``typing_extensions`` - package. - -Typed Database --------------- - -You can also use the ``TypedDict`` class to declare that all documents in a database -match a well-defined schema, as shown in the following example: - -.. code-block:: python - :emphasize-lines: 9 - - >>> from typing import TypedDict - >>> from pymongo import MongoClient - >>> from pymongo.database import Database - >>> class Movie(TypedDict): - ... name: str - ... year: int - ... - >>> client: MongoClient = MongoClient() - >>> db: Database[Movie] = client.test - >>> collection = db.test - >>> inserted = collection.insert_one({"name": "Jurassic Park", "year": 1993}) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> assert result["year"] == 1993 - -Typed Command -------------- - -When using the the ``~pymongo.database.Database.command()`` method, you can specify -the document type by providing a custom ``~bson.codec_options.CodecOptions``: - -.. code-block:: python - :emphasize-lines: 3, 5 - - >>> from pymongo import MongoClient - >>> from bson.raw_bson import RawBSONDocument - >>> from bson import CodecOptions - >>> client: MongoClient = MongoClient() - >>> options = CodecOptions(RawBSONDocument) - >>> result = client.admin.command("ping", codec_options=options) - >>> assert isinstance(result, RawBSONDocument) - -Custom ``collections.abc.Mapping`` subclasses and ``~typing.TypedDict`` -are also supported document types. Use the following syntax for ``~typing.TypedDict``: - -.. code-block:: python - - options: CodecOptions[MyTypedDict] = CodecOptions(...) - -Typed BSON Decoding -------------------- - -You can specify the document type returned by BSON-decoding functions by providing -``~bson.codec_options.CodecOptions``: - -.. code-block:: python - :emphasize-lines: 2,7,9-10 - - >>> from typing import Any, Dict - >>> from bson import CodecOptions, encode, decode - >>> class MyDict(Dict[str, Any]): - ... def foo(self): - ... return "bar" - ... - >>> options = CodecOptions(document_class=MyDict) - >>> doc = {"x": 1, "y": 2} - >>> bsonbytes = encode(doc, codec_options=options) - >>> rt_document = decode(bsonbytes, codec_options=options) - >>> assert rt_document.foo() == "bar" - -``~bson.raw_bson.RawBSONDocument`` and ``~typing.TypedDict`` are also supported document -types. Use the following syntax for ``~typing.TypedDict``: - -.. code-block:: python - - options: CodecOptions[MyTypedDict] = CodecOptions(...) - -Troubleshooting ---------------- - -Client Type Annotations -~~~~~~~~~~~~~~~~~~~~~~~ - -If you don't add a type annotation for a ``~pymongo.mongo_client.MongoClient`` object, -you might see the following Mypy error: - -.. code-block:: python - - from pymongo import MongoClient - client = MongoClient() # error: Need type annotation for "client" - -The solution is to annotate the type as -``client: MongoClient`` or ``client: MongoClient[Dict[str, Any]]``. - -Incompatible Types -~~~~~~~~~~~~~~~~~~ - -If you use the generic form of ``~pymongo.mongo_client.MongoClient``, you -might see the following Mypy error: - -.. code-block:: python - - from pymongo import MongoClient - - client: MongoClient = MongoClient() - client.test.test.insert_many( - {"a": 1} - ) # error: Dict entry 0 has incompatible type "str": "int"; - # expected "Mapping[str, Any]": "int" - -The solution is to declare the ``MongoClient`` as shown below: - -.. code-block:: python - - ``client: MongoClient[Dict[str, Any]]`` - -You might also see an ``incompatible type`` error if you pass a list to the -``insert_one()`` method, as shown in the following example: - -.. code-block:: python - - from pymongo import MongoClient - from typing import Mapping - client: MongoClient = MongoClient() - client.test.test.insert_one( - [{}] - ) # error: Argument 1 to "insert_one" of "Collection" has - # incompatible type "List[Dict[, ]]"; - # expected "Mapping[str, Any]" - -The solution is to pass a document, rather than a list, to the ``insert_one({})`` method. - -Modifying Raw BSON -~~~~~~~~~~~~~~~~~~ - -Instances of the ``~bson.raw_bson.RawBSONDocument`` class are read-only. -The following example shows the error you will see if you try to set a value on a -``RawBSONDocument`` object: - -.. code-block:: python - - from bson.raw_bson import RawBSONDocument - from pymongo import MongoClient - - client = MongoClient(document_class=RawBSONDocument) - coll = client.test.test - doc = {"my": "doc"} - coll.insert_one(doc) - retrieved = coll.find_one({"_id": doc["_id"]}) - assert retrieved is not None - assert len(retrieved.raw) > 0 - retrieved[ - "foo" - ] = "bar" # error: Unsupported target for indexed assignment - # ("RawBSONDocument") [index] diff --git a/source/includes/type-hints/intro.rst b/source/includes/type-hints/intro.rst new file mode 100644 index 00000000..4aa6a0b6 --- /dev/null +++ b/source/includes/type-hints/intro.rst @@ -0,0 +1,6 @@ +If your application uses Python 3.5 or later, you can add *type hints*, +as described in `PEP 484 `__, to your code. +Type hints denote the data types of variables, parameters, and function return +values, and the structure of documents. +Some IDEs can use type hints to check your code for type errors and suggest +appropriate options for code completion. \ No newline at end of file diff --git a/source/includes/type-hints/tip-type-checkers.rst b/source/includes/type-hints/tip-type-checkers.rst new file mode 100644 index 00000000..46f73571 --- /dev/null +++ b/source/includes/type-hints/tip-type-checkers.rst @@ -0,0 +1,4 @@ +.. tip:: Type-Checking Tools + + To learn more about type-checking tools available for Python, see + :ref:`Type Checkers ` on the Tools page. \ No newline at end of file diff --git a/source/includes/type-hints/troubleshooting-client-type.rst b/source/includes/type-hints/troubleshooting-client-type.rst new file mode 100644 index 00000000..85e9c2e1 --- /dev/null +++ b/source/includes/type-hints/troubleshooting-client-type.rst @@ -0,0 +1,13 @@ +Client Type Annotations +~~~~~~~~~~~~~~~~~~~~~~~ + +If you don't add a type annotation for your ``MongoClient`` object, +your type checker might show an error similar to the following: + +.. code-block:: python + + from pymongo import MongoClient + client = MongoClient() # error: Need type annotation for "client" + +The solution is to annotate the ``MongoClient`` object as +``client: MongoClient`` or ``client: MongoClient[Dict[str, Any]]``. \ No newline at end of file diff --git a/source/includes/type-hints/troubleshooting-incompatible-type.rst b/source/includes/type-hints/troubleshooting-incompatible-type.rst new file mode 100644 index 00000000..dc7e8fbd --- /dev/null +++ b/source/includes/type-hints/troubleshooting-incompatible-type.rst @@ -0,0 +1,17 @@ +Incompatible Type +~~~~~~~~~~~~~~~~~ + +If you specify ``MongoClient`` as a type hint but don't include data types for +the document, keys, and values, your type checker might show an error similar to +the following: + +.. code-block:: python + + error: Dict entry 0 has incompatible type "str": "int"; + expected "Mapping[str, Any]": "int" + +The solution is to add the following type hint to your ``MongoClient`` object: + +.. code-block:: python + + ``client: MongoClient[Dict[str, Any]]`` \ No newline at end of file diff --git a/source/includes/type-hints/typeddict-availability.rst b/source/includes/type-hints/typeddict-availability.rst new file mode 100644 index 00000000..2898e590 --- /dev/null +++ b/source/includes/type-hints/typeddict-availability.rst @@ -0,0 +1,5 @@ +The `TypedDict `__ class +is in the ``typing`` module, which +is available only in Python 3.8 and later. To use the ``TypedDict`` class in +earlier versions of Python, install the +`typing_extensions `__ package. \ No newline at end of file diff --git a/source/includes/write/bulk-write.py b/source/includes/write/bulk-write.py index 82793cf7..1c6c36f5 100644 --- a/source/includes/write/bulk-write.py +++ b/source/includes/write/bulk-write.py @@ -10,6 +10,17 @@ ) # end-bulk-insert-one +# start-bulk-insert-one-typed +class Restaurant (TypedDict): + name: str + cuisine: str + borough: str + restaurant_id: str + +operation = pymongo.InsertOne(Restaurant( + name="Mongo's Deli", cuisine="Sandwiches", borough="Manhattan", restaurant_id="1234")) +# end-bulk-insert-one-typed + # start-bulk-update-one operation = UpdateOne( namespace="sample_restaurants.restaurants", @@ -39,6 +50,19 @@ ) # end-bulk-replace-one +# start-bulk-replace-one-typed +class Restaurant (TypedDict): + name: str + cuisine: str + borough: str + restaurant_id: str + +operation = pymongo.ReplaceOne( + { "restaurant_id": "1234" }, + Restaurant(name="Mongo's Pizza", cuisine="Pizza", borough="Brooklyn", restaurant_id="5678") +) +# end-bulk-replace-one-typed + # start-bulk-delete-one operation = DeleteOne( namespace="sample_restaurants.restaurants", diff --git a/source/run-command.txt b/source/run-command.txt index 2c33550e..22296294 100644 --- a/source/run-command.txt +++ b/source/run-command.txt @@ -223,6 +223,66 @@ The output of this command includes information about the collections in the database, and describes the amount and size of data stored across collections. +Type Hints +---------- + +The ``Database.command()`` method can decode the returned BSON documents to instances +of a specific class. To specify this class, construct a ``CodecOptions`` object and pass +the class name. The class can be one of the following types: + +- ``bson.raw_bson.RawBSONDocument``. To learn more about the ``RawBSONDocument`` class, + see :ref:`pymongo-bson-raw`. +- A subclass of the ``collections.abc.Mapping`` type, such as ``OrderedDict``. + Depending on the strictness of your type-checking rules, you might also need to + specify types for the key and value. +- A subclass of the ``TypedDict`` type. To pass a ``TypedDict`` subclass for this + parameter, you must also include the class in a type hint for your ``CodecOptions`` + object. + +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst + +The following example decodes the BSON returned by the ``ping`` command to instances +of the ``RawBSONDocument`` class: + +.. code-block:: python + :emphasize-lines: 3, 6-7 + + from pymongo import MongoClient + from bson.raw_bson import RawBSONDocument + from bson import CodecOptions + + client: MongoClient = MongoClient() + options = CodecOptions(RawBSONDocument) + result = client.admin.command("ping", codec_options=options) + +To decode BSON to a subclass of the ``TypedDict`` class, specify the class name in +the ``CodecOptions`` type hint, as shown in the following example: + +.. code-block:: python + :emphasize-lines: 4, 6-8, 11 + + from pymongo import MongoClient + from bson.raw_bson import RawBSONDocument + from bson import CodecOptions + from typing import TypedDict + + class Movie(TypedDict): + name: str + year: int + + client: MongoClient = MongoClient() + options: CodecOptions[Movie] = CodecOptions(Movie) + result = client.admin.command("ping", codec_options=options) + +Troubleshooting +--------------- + +.. include:: /includes/type-hints/troubleshooting-client-type.rst + +.. include:: /includes/type-hints/troubleshooting-incompatible-type.rst + .. _pymongo-addtl-info-runcommand: Additional Information diff --git a/source/tools.txt b/source/tools.txt index bf70301a..d3516d83 100644 --- a/source/tools.txt +++ b/source/tools.txt @@ -260,6 +260,27 @@ daemon in the global application group: and in the `Multiple Python Sub Interpreters `__ section of the mod_wsgi documentation. +Type Checkers +------------- + +For a list of tools that can use type hints to detect errors in your code, +see `Static Typing with Python `__ +in the ``typing`` module documentation. + +.. note:: + + The default values for generic document types are not yet available in Mypy. + For a discussion of the Mypy limitations that caused this issue, see the + :github:`Mypy GitHub repository `. + + If you're using Mypy and want to opt out of using the provided types, add the + following lines to your Mypy configuration file: + + .. code-block:: python + + [mypy-pymongo] + follow_imports = False + Alternative Python Drivers -------------------------- @@ -280,5 +301,4 @@ This section lists alternatives to {+driver-short+}. `PythonAnywhere `__ does not support. For more information, see - the relevant `Jira ticket. `__ - + the relevant `Jira ticket. `__ \ No newline at end of file diff --git a/source/write/bulk-write.txt b/source/write/bulk-write.txt index ea2dd73d..4e4579b4 100644 --- a/source/write/bulk-write.txt +++ b/source/write/bulk-write.txt @@ -95,6 +95,23 @@ The following example creates an instance of ``InsertOne``: :language: python :copyable: +You can also create an instance of ``InsertOne`` by passing an instance of a custom class +to the constructor. This provides additional type safety if you're using a type-checking +tool. The instance you pass must inherit from the ``TypedDict`` class. + +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst + +The following example constructs an ``InsertOne`` instance by using a custom +class for added type safety: + +.. literalinclude:: /includes/write/bulk-write.py + :start-after: start-bulk-insert-one-typed + :end-before: end-bulk-insert-one-typed + :language: python + :copyable: + To insert multiple documents, create an instance of ``InsertOne`` for each document. .. include:: /includes/write/unique-id-note.rst @@ -157,8 +174,27 @@ The following example creates an instance of ``ReplaceOne``: :language: python :copyable: +You can also create an instance of ``ReplaceOne`` by passing an instance of a custom class +to the constructor. This provides additional type safety if you're using a type-checking +tool. The instance you pass must inherit from the ``TypedDict`` class. + +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst + +The following example constructs a ``ReplaceOne`` instance by using a custom +class for added type safety: + +.. literalinclude:: /includes/write/bulk-write.py + :start-after: start-bulk-replace-one-typed + :end-before: end-bulk-replace-one-typed + :language: python + :copyable: + To replace multiple documents, you must create an instance of ``ReplaceOne`` for each document. +.. include:: /includes/type-hints/tip-type-checkers.rst + Delete Operations ~~~~~~~~~~~~~~~~~ @@ -517,6 +553,44 @@ The ``MongoClient.bulk_write()`` method returns a ``ClientBulkWriteResult`` obje * - ``upserted_count`` - | The number of documents upserted, if any. +Type Hints +---------- + +.. include:: /includes/type-hints/intro.rst + +When you use the ``bulk_write()`` method to perform an ``InsertOne`` or ``ReplaceOne`` +operation, +The following code example shows how to insert +- ``~pymongo.collection.Collection.bulk_write()`` + +For ``bulk_write()``, both the ``~pymongo.operations.InsertOne()`` and +``~pymongo.operations.ReplaceOne()`` operators are generic. + +The following code example shows that the results are the same as the preceding examples +when you call the ``bulk_write()`` method: + +.. code-block:: python + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.operations import InsertOne + from pymongo.collection import Collection + client: MongoClient = MongoClient() + collection: Collection[Movie] = client.test.test + inserted = collection.bulk_write([InsertOne(Movie(name="Jurassic Park", year=1993))]) + result = collection.find_one({"name": "Jurassic Park"}) + assert result is not None + assert result["year"] == 1993 + +.. include:: /includes/type-hints/tip-more-info.rst + +Troubleshooting +--------------- + +.. include:: /includes/type-hints/troubleshooting-client-type.rst + +.. include:: /includes/type-hints/troubleshooting-incompatible-type.rst + Additional Information ---------------------- diff --git a/source/write/insert.txt b/source/write/insert.txt index 19ed2b42..39b41cc8 100644 --- a/source/write/insert.txt +++ b/source/write/insert.txt @@ -60,6 +60,27 @@ The following example inserts a document into the ``restaurants`` collection: sample_restaurants.restaurants.insert_one({"name" : "Mongo's Burgers"}) +You can also pass an instance of a custom class to the ``insert_one()`` method. +This provides additional type safety if you're using a type-checking +tool. The instance you pass must inherit from the ``TypedDict`` class. + +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst + +The following example passes an instance of the ``Restaurant`` class to the ``insert_one()`` +method for added type safety: + +.. code-block:: python + :copyable: true + + class Restaurant(TypedDict): + name: str + + sample_restaurants.restaurants.insert_one(Restaurant(name="Mongo's Burgers") + +.. include:: /includes/type-hints/tip-type-checkers.rst + Insert Multiple Documents ------------------------- @@ -78,6 +99,32 @@ The following example inserts a list of documents into the ``restaurants`` colle sample_restaurants.restaurants.insert_many(document_list) +You can also pass a list of instances of a custom class to the ``insert_many()`` method. +This provides additional type safety if you're using a type-checking +tool. The instances you pass must inherit from the ``TypedDict`` class. + +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst + +The following example calls the ``insert_many()`` method and passes a list that contains +instances of the ``Restaurant`` class. This adds type safety to the insert operation. + +.. code-block:: python + :copyable: true + + class Restaurant(TypedDict): + name: str + + document_list = [ + Restaurant(name="Mongo's Burgers"), + Restaurant(name="Mongo's Pizza") + ] + + sample_restaurants.restaurants.insert_many(document_list) + +.. include:: /includes/type-hints/tip-type-checkers.rst + Modify Insert Behavior ---------------------- @@ -140,6 +187,142 @@ document-level validation. sample_restaurants.restaurants.insert_many(document_list, bypass_document_validation = True) +Troubleshooting +--------------- + +.. include:: /includes/type-hints/troubleshooting-client-type.rst + +.. include:: /includes/type-hints/troubleshooting-incompatible-type.rst + +You might see a similar error if you pass a list to the ``insert_one()`` method: + +.. code-block:: bash + + error: Argument 1 to "insert_one" of "Collection" has + incompatible type "List[Dict[, ]]"; + expected "Mapping[str, Any]" + +This error occurs because the ``insert_one()`` method accepts a document, not a list. +You can resolve this error by passing a document to the ``insert_one()`` method or by +calling the ``insert_many()`` method instead. + +TypedDict Missing _id Key +~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you don't specify the ``_id`` field, {+driver-short+} automatically +inserts it into the document. +You can retrieve the value of the ``_id`` field at runtime, but if you use MyPy or another +tool to perform static type-checking, it won't find the ``_id`` field in your class and +will show an error similar to the following: + +.. code-block:: bash + + TypedDict has no key "_id" + +This is caused by code similar to the following: + +.. code-block:: python + :emphasize-lines: 13 + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.collection import Collection + class Movie(TypedDict): + name: str + year: int + + client: MongoClient = MongoClient() + collection: Collection[Movie] = client.test.test + inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) + result = collection.find_one({"name": "Jurassic Park"}) + # _id is present but was added by PyMongo; this will raise a type-checking error + assert result["_id"] + +One solution is to add a ``# type:ignore`` +comment to the end of the line that uses the ``_id`` field. This comment instructs the +type-checking tool to ignore any errors that the line causes. The following example shows +how to implement this solution; + +.. code-block:: python + :emphasize-lines: 15 + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.collection import Collection + + class Movie(TypedDict): + name: str + year: int + + collection: Collection[Movie] = client.test.test + inserted = collection.insert_one( + Movie(name="Jurassic Park", year=1993) + ) + result = collection.find_one({"name": "Jurassic Park"}) + assert result is not None + assert result["_id"] # type:ignore[typeddict-item] + +Instead of ignoring the type error, you can avoid it by including the ``_id`` field in +your model class, and explicitly specifying a value for this field when you create the +class instance. The following code shows how to implement this solution: + +.. code-block:: python + :emphasize-lines: 4,7,13 + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.collection import Collection + from bson import ObjectId + + class Movie(TypedDict): + _id: ObjectId + name: str + year: int + + collection: Collection[ExplicitMovie] = client.test.test + inserted = collection.insert_one( + ExplicitMovie(_id=ObjectId(), name="Jurassic Park", year=1993) + ) + result = collection.find_one({"name": "Jurassic Park"}) + assert result is not None + assert result["_id"] + +One drawback to adding the ``_id`` field to your custom class is that you must include a +value for the field for every instance of the class that you create. +To avoid this, you can install the +``typing.NotRequired`` package, which includes the ``NotRequired`` type hint. If you +use this type hint for the ``_id`` field, you can access the value of the ``_id`` field +at runtime without seeing any compile-time type errors. + +The following code example shows how to implement this solution: + +.. code-block:: python + + from typing import TypedDict, NotRequired + from pymongo import MongoClient + from pymongo.collection import Collection + from bson import ObjectId + + class Movie(TypedDict): + _id: NotRequired[ObjectId] + name: str + year: int + + client: MongoClient = MongoClient() + collection: Collection[Movie] = client.test.test + inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) + result = collection.find_one({"name": "Jurassic Park"}) + assert result is not None + assert result["_id"] + +.. important:: NotRequired Requires Python 3.11+ + + The `NotRequired `__ + class is available only in Python 3.11 and later. + To use ``NotRequired`` in earlier versions of Python, install the + `typing_extensions `__ + package instead. + Additional Information ----------------------