[Encryption] Add support for Field-Level Automatic and Queryable Encryption by GromNaN · Pull Request #2759 · doctrine/mongodb-odm

GromNaN · 2025-04-25T17:30:42Z

Q	A
Type	feature
BC Break	no
Fixed issues	PHPORM-317

Summary

⚠️ This feature requires a MongoDB Atlas or Enterprise license.

Client-Side Field Level Encryption (CSFLE) is a feature that enables you to encrypt data in your application before you send it over the network to MongoDB. With CSFLE enabled, no MongoDB product has access to your data in an unencrypted form.

Queryable Encryption gives you the ability to perform the following tasks:

Encrypt sensitive data fields from the client-side.

Store sensitive data fields as fully randomized encrypted data on the database server-side.

Run expressive queries on the encrypted data.

Implementation:

Add #[Encrypt] attribute with field encryption options.
Add autoEncryption configuration with options to create an encryption client (MongoDB collection that stores encryption keys for each field)

The encrypted collection must be created using the schema manager (odm:schema:create command)

alcaeus

Left some questions, and phpstan seems to be unhappy about some of the mapping types.

The logic and tests for it look solid.

docs/en/reference/attributes-reference.rst

doctrine-mongo-mapping.xsd

lib/Doctrine/ODM/MongoDB/DocumentManager.php

lib/Doctrine/ODM/MongoDB/Mapping/Annotations/Encrypt.php

alcaeus · 2025-05-07T11:49:29Z

doctrine-mongo-mapping.xsd

+      <xs:element name="field" type="odm:field" minOccurs="0" maxOccurs="unbounded" />
+      <xs:element name="embed-one" type="odm:embed-one" minOccurs="0" maxOccurs="unbounded" />
+      <xs:element name="embed-many" type="odm:embed-many" minOccurs="0" maxOccurs="unbounded" />
+      <xs:element name="encrypt" type="odm:encrypt-field" minOccurs="0" maxOccurs="1" />


Why is this an element instead of an attribute? Is it to future-proof this in case there will be options for encrypting entire documents in future?

Yes, I plan to add a "key" attribute to the element.

As discussed, the "key" don't need to be configured with queryable encryption. I keep the distinct XML attribute for extensibility.

alcaeus · 2025-05-07T11:53:16Z

lib/Doctrine/ODM/MongoDB/SchemaManager.php

+                $this->dm->getClientEncryption(),
+                $this->dm->getConfiguration()->getKmsProvider(),
+                null, // @todo when is it necessary to set the master key?
+                $options,


Should you be using $this->getWriteOptions here like below and add the encrypted field config?

tests/Doctrine/ODM/MongoDB/Tests/Functional/QueryableEncryptionTest.php

GromNaN · 2025-05-22T15:02:00Z

lib/Doctrine/ODM/MongoDB/Mapping/Driver/XmlDriver.php

+                            'min', 'max' => match ($mapping['type']) {
+                                Type::INT => (int) $encryptValue,
+                                Type::FLOAT => (float) $encryptValue,
+                                Type::DECIMAL128 => new Decimal128((string) $encryptValue),
+                                Type::DATE, Type::DATE_IMMUTABLE => new UTCDateTime(new DateTimeImmutable((string) $encryptValue)),
+                                default => null, // Invalid
+                            },


I've added the transformation of values from string to native BSON types here; should use the Type::convertToDatabaseValue() inside EncryptionFieldMap.

…ngodb 2.0 (doctrine#2770)

…octrine#2773) CommandSucceededEvent|CommandFailedEvent::getServer() has been deprecated in ext-mongodb 1.20 and removed in 2.0.0. For backward compatibility: we keep this method in MongoDB ODM, but throw an exception the extension is not compatible. I'm not adding a trigger_deprecation as this is already triggered by the extension method call. The new methods getPort() and getHost() are added as proxy to extension methods that are always available in the required ext-mongodb 1.21+. This is not covered by tests, I can add tests on CommandLogger before merging.

* CursorInterface always extends Traversable, no need to assert on SPL Iterator * Remove duplicate assert

https://github.com/mongodb/docs/blob/master/source/includes/qe-tutorials/node/queryable-encryption-tutorial.js

alcaeus · 2025-06-12T12:49:29Z

lib/Doctrine/ODM/MongoDB/SchemaManager.php

+                $class->getCollection(),
+                $this->dm->getClientEncryption(),
+                $this->dm->getConfiguration()->getKmsProvider(),
+                null, // @todo when is it necessary to set the master key?


To leave a quick answer, some KMS providers require a master key (e.g. AWS)

alcaeus · 2025-06-12T12:50:04Z

tests/Doctrine/ODM/MongoDB/Tests/BaseTestCase.php

+            new Command(['buildInfo' => 1]),
+        )->toArray()[0];
+
+        if (! in_array('enterprise', $buildInfo->modules ?? [])) {


I noticed that atlas-local does not expose an enterprise module, so we may want to remove this check before wrapping up the feature.

* Add test document classes from tutorial https://github.com/mongodb/docs/blob/master/source/includes/qe-tutorials/node/queryable-encryption-tutorial.js * Create encryption field map * Create encrypted collection * Use promoted properties for encrypted document tests * Revert changes in BaseTestCase * Rename EncryptionFieldMapTest * Create Configuration::getDriverOptions() to create the client * Support class-level #[Encrypt] attribute * Skip QE tests on non-supported server configuration * Add documentation on #[Encrypt] attribute * Add XML mapping for "encrypt" tag * Create specific xsd type for embedded documents to enable <encrypt> tag * Fix import class Throwable * Fix access to $version property before initialization * Use an enum for query type * Make getClientEncryption internal * Improve type of min/max bounds for range queries * Use getWriteOptions with createEncryptedCollection * Use random local master key * Add assertion on non-decrypted data * Ignore all DOCTRINE_MONGODB_DATABASE phpstan errors * Baseline phpstan * Fix CS and skip phpstan issues

GromNaN force-pushed the fle branch from 309178e to f70072c Compare April 28, 2025 07:49

GromNaN changed the base branch from 2.11.x to 2.12.x April 28, 2025 20:08

GromNaN force-pushed the fle branch 2 times, most recently from a6ec9ed to 81f3e90 Compare April 30, 2025 14:46

GromNaN changed the base branch from 2.12.x to feature/queryable-encryption May 5, 2025 13:48

alcaeus reviewed May 7, 2025

View reviewed changes

GromNaN commented May 7, 2025

View reviewed changes

tests/Doctrine/ODM/MongoDB/Tests/Functional/QueryableEncryptionTest.php Outdated Show resolved Hide resolved

Make XSD more strict for other Document types (doctrine#2766)

45bb9eb

GromNaN force-pushed the fle branch 5 times, most recently from f67e094 to 18d7c18 Compare May 22, 2025 13:56

GromNaN commented May 22, 2025

View reviewed changes

GromNaN force-pushed the fle branch from 5da526a to ac78f65 Compare May 22, 2025 21:43

GromNaN added 2 commits May 23, 2025 09:27

Catch BulkWriteException, since WriteException is removed from ext-mo…

9cc8291

…ngodb 2.0 (doctrine#2770)

Merge release 2.11.2 into 2.12.x (doctrine#2772)

3783f0e

GromNaN force-pushed the fle branch from ac78f65 to 9e5d36d Compare May 23, 2025 14:04

GromNaN added 12 commits May 26, 2025 14:18

Remove unnecessary assert calls (doctrine#2775)

b0909ca

* CursorInterface always extends Traversable, no need to assert on SPL Iterator * Remove duplicate assert

CSFLEQE

d41e239

Add test document classes from tutorial

5cb3c26

https://github.com/mongodb/docs/blob/master/source/includes/qe-tutorials/node/queryable-encryption-tutorial.js

Create encryption field map

af4db00

Create encrypted collection

195bbae

Use promoted properties for encrypted document tests

c007af3

Revert changes in BaseTestCase

8aeeb5f

Rename EncryptionFieldMapTest

a0c417c

Create Configuration::getDriverOptions() to create the client

a2fc8d1

Support class-level #[Encrypt] attribute

95a9d40

Skip QE tests on non-supported server configuration

db63ef9

GromNaN added 14 commits June 12, 2025 13:57

Add documentation on #[Encrypt] attribute

9ac90f1

Add XML mapping for "encrypt" tag

cbaa9ae

Create specific xsd type for embedded documents to enable <encrypt> tag

bf3ffff

Fix import class Throwable

98163d6

Fix access to $version property before initialization

f31b629

Use an enum for query type

3ee9a68

Make getClientEncryption internal

258e06b

Improve type of min/max bounds for range queries

0985cae

Use getWriteOptions with createEncryptedCollection

1a1b93d

Use random local master key

799734b

Add assertion on non-decrypted data

90782b2

Ignore all DOCTRINE_MONGODB_DATABASE phpstan errors

eade8d5

Baseline phpstan

0376f22

Fix CS and skip phpstan issues

ad2d30c

GromNaN force-pushed the fle branch from 9e5d36d to ad2d30c Compare June 12, 2025 11:58

GromNaN marked this pull request as ready for review June 12, 2025 12:27

GromNaN requested a review from alcaeus June 12, 2025 12:40

alcaeus approved these changes Jun 12, 2025

View reviewed changes

GromNaN merged commit 72ee408 into doctrine:feature/queryable-encryption Jun 12, 2025
20 checks passed

GromNaN mentioned this pull request Jun 12, 2025

PHPORM-13 Feature Queryable Encryption #2779

Merged

10 tasks

GromNaN changed the title ~~Add support for Field-Level Automatic and Queryable Encryption~~ [Encryption] Add support for Field-Level Automatic and Queryable Encryption Jul 29, 2025

GromNaN deleted the fle branch July 30, 2025 16:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Encryption] Add support for Field-Level Automatic and Queryable Encryption#2759

[Encryption] Add support for Field-Level Automatic and Queryable Encryption#2759
GromNaN merged 29 commits intodoctrine:feature/queryable-encryptionfrom
GromNaN:fle

GromNaN commented Apr 25, 2025 •

edited

Loading

Uh oh!

alcaeus left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alcaeus May 7, 2025

Uh oh!

GromNaN May 7, 2025

Uh oh!

GromNaN Jun 12, 2025

Uh oh!

alcaeus May 7, 2025

Uh oh!

Uh oh!

GromNaN May 22, 2025

Uh oh!

alcaeus Jun 12, 2025

Uh oh!

alcaeus Jun 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

GromNaN commented Apr 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

alcaeus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alcaeus May 7, 2025

Choose a reason for hiding this comment

Uh oh!

GromNaN May 7, 2025

Choose a reason for hiding this comment

Uh oh!

GromNaN Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

alcaeus May 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

GromNaN May 22, 2025

Choose a reason for hiding this comment

Uh oh!

alcaeus Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

alcaeus Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

GromNaN commented Apr 25, 2025 •

edited

Loading