Skip to content

PYTHON-5121 - Use canonical Extended JSON for BSON binary vector spec… #2215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 24, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions bson/json_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -507,6 +507,29 @@ def loads(s: Union[str, bytes, bytearray], *args: Any, **kwargs: Any) -> Any:
return json.loads(s, *args, **kwargs)


def load(fp: Any, *args: Any, **kwargs: Any) -> Any:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's open a new ticket for this. Adding this increases the scope of this ticket too much and opens other questions like "do we also need json_util.dump?"

I also question if this is valuable enough to add since it's pretty simple to use json_util.loads:

json_util.loads(fp.read())

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point--I don't think we need json_util.load() when we can use json_util.loads(fp.read()).

"""Helper function that wraps :func:`json.load`.

Automatically passes the object_hook for BSON type conversion.

Raises ``TypeError``, ``ValueError``, ``KeyError``, or
:exc:`~bson.errors.InvalidId` on invalid MongoDB Extended JSON.

:param json_options: A :class:`JSONOptions` instance used to modify the
decoding of MongoDB Extended JSON types. Defaults to
:const:`DEFAULT_JSON_OPTIONS`.

.. versionadded:: 4.12
"""
json_options = kwargs.pop("json_options", DEFAULT_JSON_OPTIONS)
# Execution time optimization if json_options.document_class is dict
if json_options.document_class is dict:
kwargs["object_hook"] = lambda obj: object_hook(obj, json_options)
else:
kwargs["object_pairs_hook"] = lambda pairs: object_pairs_hook(pairs, json_options)
return json.load(fp, *args, **kwargs)


def _json_convert(obj: Any, json_options: JSONOptions = DEFAULT_JSON_OPTIONS) -> Any:
"""Recursive helper method that converts BSON types so they can be
converted into json.
Expand Down
2 changes: 1 addition & 1 deletion test/bson_binary_vector/float32.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
{
"description": "Infinity Vector FLOAT32",
"valid": true,
"vector": ["-inf", 0.0, "inf"],
"vector": [{"$numberDouble": "-Infinity"}, 0.0, {"$numberDouble": "Infinity"} ],
"dtype_hex": "0x27",
"dtype_alias": "FLOAT32",
"padding": 0,
Expand Down
7 changes: 2 additions & 5 deletions test/test_bson_binary_vector.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
from pathlib import Path
from test import unittest

from bson import decode, encode
from bson import decode, encode, json_util
from bson.binary import Binary, BinaryVectorDtype

_TEST_PATH = Path(__file__).parent / "bson_binary_vector"
Expand Down Expand Up @@ -62,9 +62,6 @@ def run_test(self):
cB_exp = binascii.unhexlify(canonical_bson_exp.encode("utf8"))
decoded_doc = decode(cB_exp)
binary_obs = decoded_doc[test_key]
# Handle special float cases like '-inf'
if dtype_exp in [BinaryVectorDtype.FLOAT32]:
vector_exp = [float(x) for x in vector_exp]

# Test round-tripping canonical bson.
self.assertEqual(encode(decoded_doc), cB_exp, description)
Expand Down Expand Up @@ -104,7 +101,7 @@ def run_test(self):
def create_tests():
for filename in _TEST_PATH.glob("*.json"):
with codecs.open(str(filename), encoding="utf-8") as test_file:
test_method = create_test(json.load(test_file))
test_method = create_test(json_util.load(test_file))
setattr(TestBSONBinaryVector, "test_" + filename.stem, test_method)


Expand Down
Loading