Skip to content

Conversation

@baileympearson
Copy link

@baileympearson baileympearson commented Dec 26, 2024

Summary

This PR adds support for declaring encrypted schemas, as the first step in adding first class support for CSFLE to mongoose. Functionally:

  1. A new schema option, encryptionType, has been added. This is required for schemas that are declaring encrypted fields, and it determines whether the schema will be configured for 'csfle' or 'qe'.
  2. Keys in schemas can now be configured with an additional encrypt option. This option contains metadata for libmongocrypt to encrypt the field (which will be automatically included in a schemaMap or encryptedFieldsMap). The contents of this document are exactly the same as the fields used to configure a field for csfle or qe, except that bsonType is not required (inferred from the schema type).

This PR also updates all the schema modifiers / cloning methods to account for updating encrypted fields as well.

Examples

Declare an encrypted schema:

const encryptedSchema = new Schema({
  name: { type: String, encrypt: { keyId: '...' } },
  friend: { 
      // nested objects allowed
      name: { type: String, encrypt: { keyId: '...' } },
  },
  // arrays allowed - but becase of how CSFLE/QE encrypted arrays, the arrays are encrypted as a whole
  books: { type: [String], encrypt: { keyId: '...' } }
}, { encryptionType: 'qe' });

Modify / clone encrypted schemas

encryptedSchema.clone();

encryptedSchema.add({ name: String }) // name is no longer encrypted
encryptedSchema.add({ newKey: { type: String, encrypt: { keyId: '...' } }); // new encrypted key added

encryptedSchema.remove('name'); // name no longer in schema at all

encryptedSchema.pick('friends'); // returns a new schema with only one field, `friends`, that is encrypted.

encryptedSchema.omit('friends'); // returns a new schema with all fields except `frields`

@baileympearson baileympearson marked this pull request as draft December 26, 2024 18:59
@baileympearson baileympearson force-pushed the NODE-6506-add-support-for-encrypted-schemas branch from d2412a6 to 6c469ec Compare January 11, 2025 22:07
@baileympearson baileympearson changed the title feat(NODE-xxxx): Add support for encrypted schemas feat(NODE-6506): Add support for encrypted schemas Jan 13, 2025
@baileympearson baileympearson force-pushed the NODE-6506-add-support-for-encrypted-schemas branch 3 times, most recently from 027d005 to f00f123 Compare January 13, 2025 16:20
@baileympearson baileympearson marked this pull request as ready for review January 13, 2025 16:41
@nbbeeken nbbeeken self-assigned this Jan 15, 2025
* @param {string} path
* @returns
*/
function inferBSONType(schema, path) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we instead add these as a getBSONType() method on SchemaType class? That would make this more extensible for custom schematypes.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can instead make it a getter on SchemaType - but this isn't intended to be extensible for custom schema types (see the When a schema is instantiated with a custom schema type plugin test).

FLE has pretty strict requirements about which bson types are supported and I intentionally only supported the exact schema type that correspond to FLE's supported schema types. Allowing users to provide custom schema types that get auto encrypted seems like it could confusion for users and opaque errors. A hypothetical scenario: a custom schema type sometimes casts to a support BSON type, sometimes it doesn't, causing FLE to throw only sometimes for users.

I'm open to reconsidering though - we could instead support any SchemaType that casts to exactly one supported BSON type. I'm not sure we could enforce this programmatically but documentation would probably suffice. Would you prefer that approach?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would prefer the more extensible approach just in case developers have their own custom type wrappers around built-in types. For example, users may still use mongoose-int32 for int32 support, or may have their own custom Price type that gets stored as an int32 under the hood. Using custom types like this isn't a pattern that the Mongoose docs recommend, but people do all sorts of surprising things.

@@ -0,0 +1,72 @@
'use strict';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, please name this file lib/encryptionUtils.js, Mongoose camelCases file names

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this apply to test files too?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@baileympearson baileympearson force-pushed the NODE-6506-add-support-for-encrypted-schemas branch 2 times, most recently from 86ae2c0 to d02dfc4 Compare January 23, 2025 15:41
@baileympearson baileympearson changed the base branch from master to 8.10 January 23, 2025 15:42
@baileympearson baileympearson force-pushed the NODE-6506-add-support-for-encrypted-schemas branch 2 times, most recently from f912ccf to 64b3e7e Compare February 27, 2025 20:06
@baileympearson baileympearson force-pushed the NODE-6506-add-support-for-encrypted-schemas branch from 9099d21 to 3fbefcc Compare February 27, 2025 21:01
@baileympearson baileympearson changed the base branch from 8.10 to master February 27, 2025 21:01
@baileympearson baileympearson dismissed aditi-khare-mongoDB’s stale review February 28, 2025 14:22

Aditi no longer works at MongoDB.

@baileympearson baileympearson requested review from vkarpov15 and removed request for vkarpov15 February 28, 2025 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants