INTPYTHON-527 Add Queryable Encryption support #329

aclark4life · 2025-06-27T02:16:28Z

Previous attempts and additional context here:

INTPYTHON-527 Add Queryable Encryption config #318
- Branched from transaction support and includes only helpers
INTPYTHON-527 Add queryable encryption support #319
- In which this bug in libmongocrypt was discovered
- A fix for that bug in PyMongo is being discussed here.
INTPYTHON-527 Add Queryable Encryption support #323
- Before I understood that the shared lib for mongocrypt was included in the pymongocrypt wheel.

tests/encryption_/tests.py

django_mongodb_backend/models.py

django_mongodb_backend/encryption.py

django_mongodb_backend/schema.py

django_mongodb_backend/base.py

django_mongodb_backend/fields/encryption.py

django_mongodb_backend/schema.py

django_mongodb_backend/features.py

django_mongodb_backend/routers.py

django_mongodb_backend/schema.py

django_mongodb_backend/fields/encryption.py

django_mongodb_backend/schema.py

tests/encryption_/tests.py

tests/encryption_/routers.py

tests/encryption_/models.py

tests/backend_/test_features.py

django_mongodb_backend/schema.py

tests/encryption_/models.py

django_mongodb_backend/schema.py

django_mongodb_backend/encryption.py

tests/encryption_/routers.py

django_mongodb_backend/schema.py

django_mongodb_backend/models.py

django_mongodb_backend/management/commands/get_encrypted_fields_map.py

docs/source/ref/encryption.rst

django_mongodb_backend/fields/encryption.py

django_mongodb_backend/management/commands/get_encrypted_fields_map.py

django_mongodb_backend/schema.py

tests/encryption_/tests.py

There is no v7 shared library available for Ubuntu 24.04 (ubuntu-latest)

- Rename vars used for recursive calls - parent_model -> path_prefix - new_parent_model -> path - Remove check for "fields" in encrypted_fields - encrypted_fields dict will always have "fields" key - Document queryset limitations

- MongoDB 7 has rangePreview which we do not plan to support

- Add mongocryptd to atlas CI - Doc updates - Remove custom db_type, not needed after mongodb#414 - SupportsQueryableEncryptionTests require MongoDB 8 - Update test model names prefix and suffix

django_mongodb_backend/schema.py

- Isolate test failures - admin_scripts - check_framework - migrations - test_runner - user_commands - Add support for EncryptedEmbeddedModelArrayField - Refactor - Factor _get_data_key from _get_encrypted_fields - Refactor for DRY in _get_encrypted_fields method with _field_dict helper

django_mongodb_backend/schema.py

Ruff killed the nested if statements

django_mongodb_backend/schema.py

tests/encryption_/test_fields.py

django_mongodb_backend/schema.py

- Remove extra check for auto_encryption_opts - Earlier check should suffice - Remove Encrypted* class name prefix - We know these are the QE tests - Move create data key back to _get_encrypted_fields - Remove getattr for client_encryption - Remove EmbeddedModelArrayField

aclark4life commented Jun 27, 2025

View reviewed changes

tests/encryption_/tests.py Outdated Show resolved Hide resolved

aclark4life commented Jun 27, 2025

View reviewed changes

django_mongodb_backend/models.py Outdated Show resolved Hide resolved

aclark4life commented Jun 27, 2025

View reviewed changes

django_mongodb_backend/encryption.py Outdated Show resolved Hide resolved

timgraham reviewed Jun 27, 2025

View reviewed changes

This comment was marked as resolved.

Sign in to view

timgraham reviewed Jun 30, 2025

View reviewed changes

django_mongodb_backend/features.py Outdated Show resolved Hide resolved

timgraham reviewed Jul 1, 2025

View reviewed changes

django_mongodb_backend/routers.py Outdated Show resolved Hide resolved

timgraham reviewed Jul 1, 2025

View reviewed changes

django_mongodb_backend/schema.py Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

timgraham reviewed Jul 4, 2025

View reviewed changes

tests/encryption_/models.py Show resolved Hide resolved

timgraham reviewed Jul 7, 2025

View reviewed changes

tests/encryption_/models.py Outdated Show resolved Hide resolved

aclark4life commented Jul 7, 2025

View reviewed changes

django_mongodb_backend/schema.py Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

timgraham reviewed Jul 9, 2025

View reviewed changes

tests/encryption_/tests.py Outdated Show resolved Hide resolved

tests/encryption_/tests.py Outdated Show resolved Hide resolved

tests/encryption_/tests.py Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

aclark4life commented Jul 11, 2025

View reviewed changes

tests/encryption_/tests.py Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

timgraham and others added 10 commits October 6, 2025 13:20

make atlas tests use encryption settings

e3da2f6

try shared library

fe790c7

try ubuntu 22.04 just to be sure

2270058

There is no v7 shared library available for Ubuntu 24.04 (ubuntu-latest)

Code review fixes

e38b496

- Rename vars used for recursive calls - parent_model -> path_prefix - new_parent_model -> path - Remove check for "fields" in encrypted_fields - encrypted_fields dict will always have "fields" key - Document queryset limitations

Update QE guide with complete Python tutorial

36c6bfd

update version added to 5.2.2

86486eb

Add tests for EncryptedFieldMixin

59865f7

Remove confusing paragraph about crypt shared

fc87e9f

Target 5.2.3 for release and require MongoDB 8

75873e9

- MongoDB 7 has rangePreview which we do not plan to support

try Mongo 8.0.15 on CI

324c959

aclark4life force-pushed the INTPYTHON-527 branch 3 times, most recently from 54bd164 to db4aadb Compare October 6, 2025 19:57

Misc updates

13ed19a

- Add mongocryptd to atlas CI - Doc updates - Remove custom db_type, not needed after mongodb#414 - SupportsQueryableEncryptionTests require MongoDB 8 - Update test model names prefix and suffix

aclark4life force-pushed the INTPYTHON-527 branch from 8b8704f to 13ed19a Compare October 6, 2025 23:19

timgraham reviewed Oct 7, 2025

View reviewed changes

django_mongodb_backend/schema.py Outdated Show resolved Hide resolved

aclark4life force-pushed the INTPYTHON-527 branch from 0cbca21 to 59825cd Compare October 7, 2025 16:26

aclark4life force-pushed the INTPYTHON-527 branch from 59825cd to b23c4f2 Compare October 7, 2025 16:40

timgraham reviewed Oct 7, 2025

View reviewed changes

django_mongodb_backend/schema.py Outdated Show resolved Hide resolved

Kill the helper

a0cd197

Ruff killed the nested if statements

aclark4life force-pushed the INTPYTHON-527 branch from e10d8dc to 96156c8 Compare October 7, 2025 20:37

timgraham reviewed Oct 7, 2025

View reviewed changes

aclark4life force-pushed the INTPYTHON-527 branch 3 times, most recently from 7e155a9 to f7846a6 Compare October 8, 2025 23:51

Misc updates

65b96b2

- Remove extra check for auto_encryption_opts - Earlier check should suffice - Remove Encrypted* class name prefix - We know these are the QE tests - Move create data key back to _get_encrypted_fields - Remove getattr for client_encryption - Remove EmbeddedModelArrayField

aclark4life force-pushed the INTPYTHON-527 branch from f7846a6 to 65b96b2 Compare October 8, 2025 23:54

aclark4life added 2 commits October 8, 2025 20:34

Add assertEncrypted to verify field data is binary

80881fa

Fails on CI only

25e7da1

INTPYTHON-527 Add Queryable Encryption support #329

Are you sure you want to change the base?

INTPYTHON-527 Add Queryable Encryption support #329

Conversation

aclark4life commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aclark4life commented Jun 27, 2025 •

edited

Loading