Skip to content

PYTHON-5508 - Add built-in DecimalEncoder and DecimalDecoder #2499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 21, 2025
Merged

PYTHON-5508 - Add built-in DecimalEncoder and DecimalDecoder #2499

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b6b9d24
PYTHON-5508 - Add built-in DecimalEncoder and DecimalDecoder
NoahStapp Aug 20, 2025
7b28e31
Fix typing
NoahStapp Aug 20, 2025
fd9b9e1
Address review
NoahStapp Aug 20, 2025
a66b82e
Fix docstrings
NoahStapp Aug 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions bson/decimal128.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,11 @@

import decimal
import struct
from decimal import Decimal
from typing import Any, Sequence, Tuple, Type, Union

from bson.codec_options import TypeDecoder, TypeEncoder

_PACK_64 = struct.Struct("<Q").pack
_UNPACK_64 = struct.Struct("<Q").unpack

Expand Down Expand Up @@ -58,6 +61,38 @@
_VALUE_OPTIONS = Union[decimal.Decimal, float, str, Tuple[int, Sequence[int], int]]


class DecimalEncoder(TypeEncoder):
"""Converts Python :class:`decimal.Decimal` to BSON :class:`Decimal128`.

.. warning:: When converting BSON data types to and from built-in data types,
the possibility of data loss is always present due to mismatches in underlying implementations.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"possibility of data loss is always present" is extremely alarming. We have tests that do auto-convert and they don't have data loss issues so I think the language is too strong. Can we identify and explain exactly where the risk is and when it would occur?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decimal128 supports up to 34 decimal digits of precision, decimal.Decimal by default supports up to 28 digit during arithmetic operations. When creating a new decimal.Decimal value, that 28 digit limit does not exist, so I don't think we need any warning here.

Copy link
Member

@ShaneHarvey ShaneHarvey Aug 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it seems like the truncation issue will be caught on the encoding side:

>>> Decimal128(decimal.Decimal('1'*34))
Decimal128('1111111111111111111111111111111111')
>>> Decimal128(decimal.Decimal('1'*35))
Traceback (most recent call last):
  File "<python-input-6>", line 1, in <module>
    Decimal128(decimal.Decimal('1'*35))
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shane/git/mongo-python-driver/bson/decimal128.py", line 218, in __init__
    self.__high, self.__low = _decimal_to_128(value)
                              ~~~~~~~~~~~~~~~^^^^^^^
  File "/Users/shane/git/mongo-python-driver/bson/decimal128.py", line 76, in _decimal_to_128
    value = ctx.create_decimal(value)
decimal.Inexact: [<class 'decimal.Inexact'>]


.. versionadded:: 4.15"""

@property
def python_type(self):
return Decimal

def transform_python(self, value):
return Decimal128(value)


class DecimalDecoder(TypeDecoder):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a docstring example showing out to use these classes?

"""Converts BSON :class:`Decimal128` to Python :class:`decimal.Decimal`.

.. warning:: When converting BSON data types to and from built-in data types,
the possibility of data loss is always present due to mismatches in underlying implementations.

.. versionadded:: 4.15"""

@property
def bson_type(self):
return Decimal128

def transform_bson(self, value):
return value.to_decimal()


def create_decimal128_context() -> decimal.Context:
"""Returns an instance of :class:`decimal.Context` appropriate
for working with IEEE-754 128-bit decimal floating point values.
Expand Down
10 changes: 10 additions & 0 deletions doc/changelog.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
Changelog
=========
Changes in Version 4.15.0 (XXXX/XX/XX)
--------------------------------------
.. warning:: When converting BSON data types to and from built-in data types, the possibility of data loss is always present
due to mismatches in underlying implementations.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we need this warning here too.


PyMongo 4.15 brings a number of changes including:

- Added :class:`bson.decimal128.DecimalEncoder` and :class:`bson.decimal128.DecimalDecoder`
to support encoding and decoding of BSON Decimal128 values to decimal.Decimal values using the TypeRegistry API.

Changes in Version 4.14.1 (2025/08/19)
--------------------------------------

Expand Down
25 changes: 2 additions & 23 deletions test/asynchronous/test_custom_types.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
from random import random
from typing import Any, Tuple, Type, no_type_check

from bson.decimal128 import DecimalDecoder, DecimalEncoder
from gridfs.asynchronous.grid_file import AsyncGridIn, AsyncGridOut

sys.path[0:0] = [""]
Expand Down Expand Up @@ -59,29 +60,7 @@
_IS_SYNC = False


class DecimalEncoder(TypeEncoder):
@property
def python_type(self):
return Decimal

def transform_python(self, value):
return Decimal128(value)


class DecimalDecoder(TypeDecoder):
@property
def bson_type(self):
return Decimal128

def transform_bson(self, value):
return value.to_decimal()


class DecimalCodec(DecimalDecoder, DecimalEncoder):
pass


DECIMAL_CODECOPTS = CodecOptions(type_registry=TypeRegistry([DecimalCodec()]))
DECIMAL_CODECOPTS = CodecOptions(type_registry=TypeRegistry([DecimalEncoder(), DecimalDecoder()]))


class UndecipherableInt64Type:
Expand Down
25 changes: 2 additions & 23 deletions test/test_custom_types.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
from random import random
from typing import Any, Tuple, Type, no_type_check

from bson.decimal128 import DecimalDecoder, DecimalEncoder
from gridfs.synchronous.grid_file import GridIn, GridOut

sys.path[0:0] = [""]
Expand Down Expand Up @@ -59,29 +60,7 @@
_IS_SYNC = True


class DecimalEncoder(TypeEncoder):
@property
def python_type(self):
return Decimal

def transform_python(self, value):
return Decimal128(value)


class DecimalDecoder(TypeDecoder):
@property
def bson_type(self):
return Decimal128

def transform_bson(self, value):
return value.to_decimal()


class DecimalCodec(DecimalDecoder, DecimalEncoder):
pass


DECIMAL_CODECOPTS = CodecOptions(type_registry=TypeRegistry([DecimalCodec()]))
DECIMAL_CODECOPTS = CodecOptions(type_registry=TypeRegistry([DecimalEncoder(), DecimalDecoder()]))


class UndecipherableInt64Type:
Expand Down
Loading