Skip to content

Commit acfd9ec

Browse files
committed
test(ClientSideEncryption): update spec tests
1 parent 3eedefc commit acfd9ec

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+712
-462
lines changed

test/functional/spec/client-side-encryption/README.rst

Lines changed: 174 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -32,16 +32,16 @@ The spec tests format is an extension of `transactions spec tests <https://githu
3232

3333
The semantics of `$$type` is that any actual value matching the BSON type indicated by the BSON type string is considered a match.
3434

35-
For example, the following matches a command_started_event for an insert of a document where `random` must be of type ``binData``:
36-
```
37-
- command_started_event:
38-
command:
39-
insert: *collection_name
40-
documents:
41-
- { random: { $$type: "binData" } }
42-
ordered: true
43-
command_name: insert
44-
```
35+
For example, the following matches a command_started_event for an insert of a document where `random` must be of type ``binData``::
36+
37+
- command_started_event:
38+
command:
39+
insert: *collection_name
40+
documents:
41+
- { random: { $$type: "binData" } }
42+
ordered: true
43+
command_name: insert
44+
4545

4646
The values of `$$type` correspond to `these documented string representations of BSON types <https://docs.mongodb.com/manual/reference/bson-types/>`_.
4747

@@ -79,10 +79,12 @@ Each YAML file has the following keys:
7979

8080
- ``local`` The local KMS provider.
8181

82-
- ``key`` A 64 byte local key.
82+
- ``key`` A 96 byte local key.
8383

8484
- ``schema_map``: Optional, a map from namespaces to local JSON schemas.
8585

86+
- ``keyVaultNamespace``: Optional, a namespace to the key vault collection. Defaults to "admin.datakeys".
87+
8688
- ``operations``: Array of documents, each describing an operation to be
8789
executed. Each document has the following fields:
8890

@@ -125,7 +127,8 @@ Then for each element in ``tests``:
125127

126128
#. Create a MongoClient using ``clientOptions``.
127129

128-
#. If ``client_side_encryption_opts`` includes ``aws`` as a KMS provider, pass in AWS credentials from the environment.
130+
#. If ``autoEncryptOpts`` includes ``aws`` as a KMS provider, pass in AWS credentials from the environment.
131+
#. If ``autoEncryptOpts`` does not include ``keyVaultNamespace``, default it to ``admin.datakeys``
129132

130133
#. Create a collection object from the MongoClient, using the ``database_name``
131134
and ``collection_name`` fields from the YAML file.
@@ -188,79 +191,184 @@ The spec test MUST be run with *and* without auth.
188191
Prose Tests
189192
===========
190193

191-
Tests for the KeyVault type are not included as part of the YAML tests. Tests are described
192-
as follows.
194+
Tests for the ClientEncryption type are not included as part of the YAML tests.
195+
196+
First, perform the setup.
197+
198+
#. Create a MongoClient without encryption enabled (referred to as ``client``).
199+
200+
#. Using ``client``, drop the collections ``admin.datakeys`` and ``db.coll``.
201+
202+
#. Create the following:
203+
204+
- A MongoClient configured with auto encryption (referred to as ``client_encrypted``)
205+
- A ``ClientEncryption`` object (referred to as ``client_encryption``)
206+
207+
Configure both objects with ``aws`` and the ``local`` KMS providers as follows:
208+
209+
.. code:: javascript
210+
211+
{
212+
"aws": { <AWS credentials> },
213+
"local": { "key": <base64 decoding of LOCAL_MASTERKEY> }
214+
}
215+
216+
Where LOCAL_MASTERKEY is the following base64:
193217

194-
#. Test creating a data key with the "local" KMS provider.
218+
.. code:: javascript
195219
196-
- Create a `KeyVault` with a "local" KMS provider.
197-
- Create a data key with the "local" KMS provider using `KeyVault.createDataKey()`.
198-
- Expect a BSON binary with subtype 4 to be returned.
199-
- Expect a `findOne` on the key vault collection with `_id` set to the returned binary to return a document.
200-
- Expect that document to have "masterKey.provider" set to "local"
220+
Mng0NCt4ZHVUYUJCa1kxNkVyNUR1QURhZ2h2UzR2d2RrZzh0cFBwM3R6NmdWMDFBMUN3YkQ5aXRRMkhGRGdQV09wOGVNYUMxT2k3NjZKelhaQmRCZGJkTXVyZG9uSjFk
201221
202-
#. Test creating a data key with the "aws" KMS provider.
222+
Configure both objects with ``keyVaultNamespace`` set to ``admin.datakeys``.
203223

204-
- Create a `KeyVault` with a "aws" KMS provider.
205-
- Create a data key with the "aws" KMS provider using `KeyVault.createDataKey()`.
206-
- Expect a BSON binary with subtype 4 to be returned.
207-
- Expect a `findOne` on the key vault collection with `_id` set to the returned binary to return a document.
208-
- Expect that document to have "masterKey.provider" set to "aws"
224+
Configure the ``MongoClient`` with the following ``schema_map``:
209225

210-
#. Test explicit encrypt and decrypt with the "local" KMS provider.
226+
.. code:: javascript
211227
212-
- Create a `KeyVault` with a "local" KMS provider.
213-
- Insert the a key document in to the key vault.
214-
- Use `KeyVault.encrypt` to encrypt the value "hello" with the following:
228+
{
229+
"db.coll": {
230+
"bsonType": "object",
231+
"properties": {
232+
"encrypted_placeholder": {
233+
"encrypt": {
234+
"keyId": "/placeholder",
235+
"bsonType": "string",
236+
"algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Random"
237+
}
238+
}
239+
}
240+
}
241+
}
215242
216-
- the algorithm "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic"
217-
- the "local" KMS provider
243+
Then, test creating and using data keys from a ``local`` KMS provider:
218244

219-
- Expect the value is equal to a known BSON binary of subtype 6
220-
- Use `KeyVault.decrypt` to decrypt the encrypted value
221-
- Expect the value is equal to the string "hello"
245+
#. Call ``client_encryption.createDataKey()`` with the ``local`` KMS provider and keyAltNames set to ``["local_altname"]``.
222246

223-
#. Test explicit encrypt and decrypt with the "aws" KMS provider.
247+
- Expect a BSON binary with subtype 4 to be returned, referred to as ``local_datakey_id``.
248+
- Use ``client`` to run a ``find`` on ``admin.datakeys`` by querying with the ``_id`` set to the ``local_datakey_id``.
249+
- Expect that exactly one document is returned with the "masterKey.provider" equal to "local".
224250

225-
- Create a `KeyVault` with a "aws" KMS provider.
226-
- Insert the a key document in to the key vault.
227-
- Use `KeyVault.encrypt` to encrypt the value "hello" with the following:
251+
#. Call ``client_encryption.encrypt()`` with the value "hello local", the algorithm ``AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic``, and the ``key_id`` of ``local_datakey_id``.
228252

229-
- the algorithm "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic"
230-
- the "aws" KMS provider
253+
- Expect the return value to be a BSON binary subtype 6, referred to as ``local_encrypted``.
254+
- Use ``client_encrypted`` to insert ``{ _id: "local", "value": <local_encrypted> }`` into ``db.coll``.
255+
- Use ``client_encrypted`` to run a find querying with ``_id`` of "local" and expect ``value`` to be "hello local".
231256

232-
- Expect the value is equal to a known BSON binary of subtype 6
233-
- Use `KeyVault.decrypt` to decrypt the encrypted value
234-
- Expect the value is equal to the string "hello"
257+
#. Call ``client_encryption.encrypt()`` with the value "hello local", the algorithm ``AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic``, and the ``key_alt_name`` of ``local_altname``.
235258

236-
#. Test explicit encrypt of invalid values.
259+
- Expect the return value to be a BSON binary subtype 6. Expect the value to exactly match the value of ``local_encrypted``.
237260

238-
- Create a `KeyVault` with either a "local" or "aws" KMS provider
239-
- Use `KeyVault.encrypt` to attempt to encrypt each BSON type with deterministic encryption.
261+
Then, repeat the above tests with the ``aws`` KMS provider:
240262

241-
- Expect a `string` to succeed. An exception MUST be thrown.
242-
- Expect a `document` to fail. An exception MUST be thrown.
243-
- Expect a `null` to fail. An exception MUST be thrown.
244-
- Expect a BSON binary subtype 6 to fail. An exception MUST be thrown.
263+
#. Call ``client_encryption.createDataKey()`` with the ``aws`` KMS provider, keyAltNames set to ``["aws_altname"]``, and ``masterKey`` as follows:
245264

246-
- Use `KeyVault.encrypt` to attempt to encrypt a document using randomized encryption.
265+
.. code:: javascript
247266
248-
- Expect a `document` to succeed.
249-
- Expect a BSON binary subtype 6 to fail. An exception MUST be thrown.
250-
- Expect a `null` to fail. An exception MUST be thrown.
267+
{
268+
region: "us-east-1",
269+
key: "arn:aws:kms:us-east-1:579766882180:key/89fcc2c4-08b0-4bd9-9f25-e30687b580d0"
270+
}
251271
252-
#. Test explicit encryption with auto decryption.
253272
254-
- Create a `KeyVault` with either a "local" or "aws" KMS provider
255-
- Use `KeyVault.encrypt` to encrypt a value.
256-
- Create a document, setting some field to the value.
257-
- Insert the document into a collection.
258-
- Find the document. Verify both the value matches the originally set value.
273+
- Expect a BSON binary with subtype 4 to be returned, referred to as ``aws_datakey_id``.
274+
- Use ``client`` to run a ``find`` on ``admin.datakeys`` by querying with the ``_id`` set to the ``aws_datakey_id``.
275+
- Expect that exactly one document is returned with the "masterKey.provider" equal to "aws".
276+
277+
#. Call ``client_encryption.encrypt()`` with the value "hello aws", the algorithm ``AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic``, and the ``key_id`` of ``aws_datakey_id``.
278+
279+
- Expect the return value to be a BSON binary subtype 6, referred to as ``aws_encrypted``.
280+
- Use ``client_encrypted`` to insert ``{ _id: "aws", "value": <aws_encrypted> }`` into ``db.coll``.
281+
- Use ``client_encrypted`` to run a find querying with ``_id`` of "aws" and expect ``value`` to be "hello aws".
282+
283+
#. Call ``client_encryption.encrypt()`` with the value "hello aws", the algorithm ``AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic``, and the ``key_alt_name`` of ``aws_altname``.
284+
285+
- Expect the return value to be a BSON binary subtype 6. Expect the value to exactly match the value of ``aws_encrypted``.
286+
287+
288+
Then, run the following final tests:
259289

260290
#. Test explicit encrypting an auto encrypted field.
261291

262-
- Create a `KeyVault` with either a "local" or "aws" KMS provider
263-
- Create a collection with a JSON Schema specifying an encrypted field.
264-
- Use `KeyVault.encrypt` to encrypt a value.
265-
- Create a document, setting the auto-encrypted field to the value.
266-
- Insert the document. Verify an exception is thrown.
292+
- Use ``client_encrypted`` to attempt to insert ``{ "encrypted_placeholder": (local_encrypted) }``
293+
- Expect an exception to be thrown, since this is an attempt to auto encrypt an already encrypted value.
294+
295+
296+
Corpus Test
297+
===========
298+
299+
The corpus test exhaustively enumerates all ways to encrypt all BSON value types. Note, the test data includes BSON binary subtype 4 (or standard UUID), which MUST be decoded and encoded as subtype 4. Run the test as follows.
300+
301+
1. Create a MongoClient without encryption enabled (referred to as ``client``).
302+
303+
2. Using ``client``, drop and create the collection ``db.coll`` configured with the included JSON schema `corpus/corpus-schema.json <corpus/corpus-schema.json>`_.
304+
305+
3. Using ``client``, drop the collection ``admin.datakeys``. Insert the documents `corpus/corpus-key-local.json <corpus/corpus-key-local.json>`_ and `corpus/corpus-key-aws.json <corpus/corpus-key-aws.json>`_.
306+
307+
4. Create the following:
308+
309+
- A MongoClient configured with auto encryption (referred to as ``client_encrypted``)
310+
- A ``ClientEncryption`` object (referred to as ``client_encryption``)
311+
312+
Configure both objects with ``aws`` and the ``local`` KMS providers as follows:
313+
314+
.. code:: javascript
315+
316+
{
317+
"aws": { <AWS credentials> },
318+
"local": { "key": <base64 decoding of LOCAL_MASTERKEY> }
319+
}
320+
321+
Where LOCAL_MASTERKEY is the following base64:
322+
323+
.. code:: javascript
324+
325+
Mng0NCt4ZHVUYUJCa1kxNkVyNUR1QURhZ2h2UzR2d2RrZzh0cFBwM3R6NmdWMDFBMUN3YkQ5aXRRMkhGRGdQV09wOGVNYUMxT2k3NjZKelhaQmRCZGJkTXVyZG9uSjFk
326+
327+
Configure both objects with ``keyVaultNamespace`` set to ``admin.datakeys``.
328+
329+
5. Load `corpus/corpus.json <corpus/corpus.json>`_ to a variable named ``corpus``. The corpus contains subdocuments with the following fields:
330+
331+
- ``kms`` is either ``aws`` or ``local``
332+
- ``type`` is a BSON type string `names coming from here <https://docs.mongodb.com/manual/reference/operator/query/type/>`_)
333+
- ``algo`` is either ``rand`` or ``det`` for random or deterministic encryption
334+
- ``method`` is either ``auto``, for automatic encryption or ``explicit`` for explicit encryption
335+
- ``identifier`` is either ``id`` or ``altname`` for the key identifier
336+
- ``allowed`` is a boolean indicating whether the encryption for the given parameters is permitted.
337+
- ``value`` is the value to be tested.
338+
339+
Create a new BSON document, named ``corpus_copied``.
340+
Iterate over each field of ``corpus``.
341+
342+
- If the field name is ``_id``, ``altname_aws`` and ``altname_local``, copy the field to ``corpus_copied``.
343+
- If ``method`` is ``auto``, copy the field to ``corpus_copied``.
344+
- If ``method`` is ``explicit``, use ``client_encryption`` to explicitly encrypt the value.
345+
346+
- Encrypt with the algorithm described by ``algo``.
347+
- If ``identifier`` is ``id``
348+
- If ``kms`` is ``local`` set the key_id to the UUID with base64 value ``LOCALAAAAAAAAAAAAAAAAA==``.
349+
- If ``kms`` is ``aws`` set the key_id to the UUID with base64 value ``AWSAAAAAAAAAAAAAAAAAAA==``.
350+
- If ``identifier`` is ``altname``
351+
- If ``kms`` is ``local`` set the key_alt_name to "local".
352+
- If ``kms`` is ``aws`` set the key_alt_name to "aws".
353+
354+
If ``allowed`` is true, copy the field and encrypted value to ``corpus_copied``.
355+
If ``allowed`` is false. verify that an exception is thrown. Copy the unencrypted value to to ``corpus_copied``.
356+
357+
358+
6. Using ``client_encrypted``, insert ``corpus_copied`` into ``db.coll``.
359+
360+
7. Using ``client_encrypted``, find the inserted document from ``db.coll`` to a variable named ``corpus_decrypted``. Since it should have been automatically decrypted, assert the document exactly matches ``corpus``.
361+
362+
8. Load `corpus/corpus_encrypted.json <corpus/corpus-encrypted.json>`_ to a variable named ``corpus_encrypted_expected``.
363+
Using ``client`` find the inserted document from ``db.coll`` to a variable named ``corpus_encrypted_actual``.
364+
365+
Iterate over each field of ``corpus_encrypted_expected`` and check the following:
366+
367+
- If the ``algo`` is ``det``, that the value exactly matches all fields in ``corpus_encrypted_actual`` with the same ``kms``, ``type``, and ``algo``.
368+
- If the ``algo`` is ``rand`` and ``allowed`` is true, that the value does not match any fields in ``corpus_encrypted_actual`` with the same ``kms`` and ``type``.
369+
- If the ``method`` is ``auto`` or ``explicit``, decrypt the value with ``client_encryption`` and validate the value exactly matches the corresponding field of ``corpus``.
370+
- If the ``allowed`` is false, validate the value exactly matches the corresponding field of ``corpus``.
371+
372+
9. Repeat steps 1-8 with a local JSON schema. I.e. amend step 4 to configure the schema on ``client_encrypted`` and ``client_encryption`` with the ``schema_map`` option.
373+
374+

test/functional/spec/client-side-encryption/aggregate.json

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
"_id": 1,
1212
"encrypted_string": {
1313
"$binary": {
14-
"base64": "AQAAAAAAAAAAAAAAAAAAAAACtsdOjHK3CkpNjAAMznkYbeR6Z+yLzCRv4zOb7VfKnJdmU0W5MD8GVODx8K+KuBoCbE1SfGfPgX6NhS/RKpQ12w==",
14+
"base64": "AQAAAAAAAAAAAAAAAAAAAAACwj+3zkv2VM+aTfk60RqhXq6a/77WlLwu/BxXFkL7EppGsju/m8f0x5kBDD3EZTtGALGXlym5jnpZAoSIkswHoA==",
1515
"subType": "06"
1616
}
1717
}
@@ -23,7 +23,7 @@
2323
"encrypt": {
2424
"keyId": "/altname",
2525
"bsonType": "string",
26-
"algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic"
26+
"algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Random"
2727
}
2828
},
2929
"encrypted_string": {
@@ -92,7 +92,7 @@
9292
},
9393
"keyMaterial": {
9494
"$binary": {
95-
"base64": "AQICAHhQNmWG2CzOm1dq3kWLM+iDUZhEqnhJwH9wZVpuZ94A8gF9FSYZL9Ze8TvTA3WBd3nmAAAAwjCBvwYJKoZIhvcNAQcGoIGxMIGuAgEAMIGoBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDLV3GHktEO8AlpsYBwIBEIB7ho0DQF7hEQPRz/8b61AHm2czX53Y9BNu5z+oyGYsoP643M58aTGsaHQzkTaAmGKlZTAEOjJkRJ4gZoabVuv4g6aJqf4k4w8pK7iIgHwMNy4nbUAqOWmqtnKpHZgy6jcFN2DzZzHIi4SNFsCkFc6Aw30ixtvqIDQPAXMW",
95+
"base64": "AQICAHhQNmWG2CzOm1dq3kWLM+iDUZhEqnhJwH9wZVpuZ94A8gEqnsxXlR51T5EbEVezUqqKAAAAwjCBvwYJKoZIhvcNAQcGoIGxMIGuAgEAMIGoBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDHa4jo6yp0Z18KgbUgIBEIB74sKxWtV8/YHje5lv5THTl0HIbhSwM6EqRlmBiFFatmEWaeMk4tO4xBX65eq670I5TWPSLMzpp8ncGHMmvHqRajNBnmFtbYxN3E3/WjxmdbOOe+OXpnGJPcGsftc7cB2shRfA4lICPnE26+oVNXT6p0Lo20nY5XC7jyCO",
9696
"subType": "00"
9797
}
9898
},
@@ -217,7 +217,7 @@
217217
"_id": 1,
218218
"encrypted_string": {
219219
"$binary": {
220-
"base64": "AQAAAAAAAAAAAAAAAAAAAAACtsdOjHK3CkpNjAAMznkYbeR6Z+yLzCRv4zOb7VfKnJdmU0W5MD8GVODx8K+KuBoCbE1SfGfPgX6NhS/RKpQ12w==",
220+
"base64": "AQAAAAAAAAAAAAAAAAAAAAACwj+3zkv2VM+aTfk60RqhXq6a/77WlLwu/BxXFkL7EppGsju/m8f0x5kBDD3EZTtGALGXlym5jnpZAoSIkswHoA==",
221221
"subType": "06"
222222
}
223223
}
@@ -324,7 +324,7 @@
324324
"_id": 1,
325325
"encrypted_string": {
326326
"$binary": {
327-
"base64": "AQAAAAAAAAAAAAAAAAAAAAACtsdOjHK3CkpNjAAMznkYbeR6Z+yLzCRv4zOb7VfKnJdmU0W5MD8GVODx8K+KuBoCbE1SfGfPgX6NhS/RKpQ12w==",
327+
"base64": "AQAAAAAAAAAAAAAAAAAAAAACwj+3zkv2VM+aTfk60RqhXq6a/77WlLwu/BxXFkL7EppGsju/m8f0x5kBDD3EZTtGALGXlym5jnpZAoSIkswHoA==",
328328
"subType": "06"
329329
}
330330
}

0 commit comments

Comments
 (0)