Support multiple databases connections with different crypto keys (#72)

* Added support for different keys based on database (#67)

* Updated tests to include digest and hash

* Refactored how we get settings

* Updated db connection setup

* Assert that query actually is using the expected keys

* Fixes for django 1.11

Author: peterfarrell

CHANGELOG.md

@@ -5,6 +5,7 @@
 * Added new DecimalFields for both public and symmetric key (#64)
 * Added new FloatFields for both public and symmetric key (#64)
 * Added new TimeFields for both public and symmetric key (#64)
+* Added support for different keys based on database (#67)
 
 ## 2.4.0
 

README.md

@@ -16,20 +16,100 @@ encrypt and decrypt data for fields.
 
 ## Installation
 
+### Install package 
+
 ```bash
 pip install django-pgcrypto-fields
 ```
 
-In your Django `settings.py`, add `pgcrypto` to `INSTALLED_APPS`:
+### Django settings
+
+Our library support different crypto keys for multiple databases by 
+defining the keys in your `DATABASES` settings.
 
+In `settings.py`:
 ```python
+import os
+BASEDIR = os.path.dirname(os.path.dirname(__file__))
+PUBLIC_PGP_KEY_PATH = os.path.abspath(os.path.join(BASEDIR, 'public.key'))
+PRIVATE_PGP_KEY_PATH = os.path.abspath(os.path.join(BASEDIR, 'private.key'))
+
+# Used by PGPPublicKeyField used by default if not specified by the db
+PUBLIC_PGP_KEY = open(PUBLIC_PGP_KEY_PATH).read()
+PRIVATE_PGP_KEY = open(PRIVATE_PGP_KEY_PATH).read()
+
+# Used by TextHMACField and PGPSymmetricKeyField if not specified by the db
+PGCRYPTO_KEY='ultrasecret'
+
+DIFF_PUBLIC_PGP_KEY_PATH = os.path.abspath(
+    os.path.join(BASEDIR, 'tests/keys/public_diff.key')
+)
+DIFF_PRIVATE_PGP_KEY_PATH = os.path.abspath(
+    os.path.join(BASEDIR, 'tests/keys/private_diff.key')
+)
+
+# And add 'pgcrypto' to `INSTALLED_APPS` to create the extension for
+# pgcrypto (in a migration).
 INSTALLED_APPS = (
     'pgcrypto',
-    # Other apps
+    # Other installed apps
 )
+
+DATABASES = {
+    # This db will use the default keys above
+    'default': {
+        'ENGINE': 'django.db.backends.postgresql_psycopg2',
+        'NAME': 'pgcryto_fields',
+        'USER': 'pgcryto_fields',
+        'PASSWORD': 'xxxx',
+        'HOST': 'psql.test.com',
+        'PORT': 5432,
+        'OPTIONS': {
+            'sslmode': 'require',
+        }
+    },
+    'diff_keys': {
+        'ENGINE': 'django.db.backends.postgresql_psycopg2',
+        'NAME': 'pgcryto_fields_diff',
+        'USER': 'pgcryto_fields_diff',
+        'PASSWORD': 'xxxx',
+        'HOST': 'psqldiff.test.com',
+        'PORT': 5432,
+        'OPTIONS': {
+            'sslmode': 'require',
+        },
+        'PGCRYPTO_KEY': 'djangorocks',
+        'PUBLIC_PGP_KEY': open(DIFF_PUBLIC_PGP_KEY_PATH, 'r').read(),
+        'PRIVATE_PGP_KEY': open(DIFF_PRIVATE_PGP_KEY_PATH, 'r').read(),
+    },
+}
+```
+
+### Generate GPG keys if using Public Key Encryption
+
+The public key is going to encrypt the message and the private key will be
+needed to decrypt the content. The following commands have been taken from the
+[pgcrypto documentation](http://www.postgresql.org/docs/devel/static/pgcrypto.html)
+(see Generating PGP Keys with GnuPG).
+
+Generating a public and a private key (The preferred key type is "DSA and Elgamal".):
+
+```bash
+$ gpg --gen-key
+$ gpg --list-secret-keys
+
+/home/bob/.gnupg/secring.gpg
+---------------------------
+sec   2048R/21 2014-10-23
+uid                  Test Key <[email protected]>
+ssb   2048R/42 2014-10-23
+
+
+$ gpg -a --export 42 > public.key
+$ gpg -a --export-secret-keys 21 > private.key
 ```
 
-## Upgrading to 2.4.0 from previous versions
+### Upgrading to 2.4.0 from previous versions
 
 The 2.4.0 version of this library received a large rewrite in order to support 
 auto-decryption when getting encrypted field data as well as the ability to filter 
@@ -43,7 +123,6 @@ your application to these items need to be removed as well:
 * `admin.PGPAdmin`
 * `aggregates.*`
 
-
 ## Fields
 
 `django-pgcrypto-fields` has 3 kinds of fields:
@@ -83,30 +162,6 @@ encrypt the data and a private key to decrypt it.
 Public and private keys can be set in settings with `PUBLIC_PGP_KEY` and
 `PRIVATE_PGP_KEY`.
 
-##### Generate GPG keys.
-
-The public key is going to encrypt the message and the private key will be
-needed to decrypt the content. The following commands have been taken from the
-[pgcrypto documentation](http://www.postgresql.org/docs/devel/static/pgcrypto.html)
-(see Generating PGP Keys with GnuPG).
-
-Generating a public and a private key:
-
-```bash
-$ gpg --gen-key
-$ gpg --list-secret-keys
-
-/home/bob/.gnupg/secring.gpg
----------------------------
-sec   2048R/21 2014-10-23
-uid                  Test Key <[email protected]>
-ssb   2048R/42 2014-10-23
-
-
-$ gpg -a --export 42 > public.key
-$ gpg -a --export-secret-keys 21 > private.key
-```
-
 #### Symmetric Key Encryption Fields
 
 Supported PGP symmetric key fields are:
@@ -121,33 +176,6 @@ Supported PGP symmetric key fields are:
 
 Encrypt and decrypt the data with `settings.PGCRYPTO_KEY` which acts like a password.
 
-### Django settings
-
-In `settings.py`:
-```python
-import os
-BASEDIR = os.path.dirname(os.path.dirname(__file__))
-PUBLIC_PGP_KEY_PATH = os.path.abspath(os.path.join(BASEDIR, 'public.key'))
-PRIVATE_PGP_KEY_PATH = os.path.abspath(os.path.join(BASEDIR, 'private.key'))
-
-
-# Used by PGPPublicKeyField
-PUBLIC_PGP_KEY = open(PUBLIC_PGP_KEY_PATH).read()
-PRIVATE_PGP_KEY = open(PRIVATE_PGP_KEY_PATH).read()
-
-# Used by TextHMACField and PGPSymmetricKeyField
-PGCRYPTO_KEY='ultrasecret'
-
-
-# And add 'pgcrypto' to `INSTALLED_APPS` to create the extension for
-# pgcrypto (in a migration).
-INSTALLED_APPS = (
-    'pgcrypto',
-    # Other installed apps
-)
-
-```
-
 ### Usage
 
 #### Model Definition

pgcrypto/fields.py

@@ -1,4 +1,3 @@
-from django.conf import settings
 from django.db import models
 
 from pgcrypto import (
@@ -12,6 +11,7 @@
 )
 from pgcrypto.mixins import (
     DecimalPGPFieldMixin,
+    get_setting,
     HashMixin,
     PGPPublicKeyFieldMixin,
     PGPSymmetricKeyFieldMixin,
@@ -23,9 +23,9 @@ class TextDigestField(HashMixin, models.TextField):
     """Text digest field for postgres."""
     encrypt_sql = DIGEST_SQL
 
-    def get_encrypt_sql(self):
+    def get_encrypt_sql(self, connection):
         """Get encrypt sql."""
-        return self.encrypt_sql.format(settings.PGCRYPTO_KEY)
+        return self.encrypt_sql.format(get_setting(connection, 'PGCRYPTO_KEY'))
 
 
 TextDigestField.register_lookup(HashLookup)

pgcrypto/mixins.py

@@ -16,21 +16,27 @@ def remove_validators(validators, validator_class):
     return [v for v in validators if not isinstance(v, validator_class)]
 
 
+def get_setting(connection, key):
+    """Get key from connection or default to settings."""
+    if key in connection.settings_dict:
+        return connection.settings_dict[key]
+    else:
+        return getattr(settings, key)
+
+
 class DecryptedCol(Col):
     """Provide DecryptedCol support without using `extra` sql."""
 
     def __init__(self, alias, target, output_field=None):
         """Init the decryption."""
-        self.decrypt_sql = target.get_decrypt_sql()
-        self.cast_sql = target.get_cast_sql()
         self.target = target
 
         super(DecryptedCol, self).__init__(alias, target, output_field)
 
     def as_sql(self, compiler, connection):
         """Build SQL with decryption and casting."""
         sql, params = super(DecryptedCol, self).as_sql(compiler, connection)
-        sql = self.decrypt_sql % (sql, self.cast_sql)
+        sql = self.target.get_decrypt_sql(connection) % (sql, self.target.get_cast_sql())
         return sql, params
 
 
@@ -67,9 +73,9 @@ def get_placeholder(self, value=None, compiler=None, connection=None):
         if value is None or value.startswith('\\x'):
             return '%s'
 
-        return self.get_encrypt_sql()
+        return self.get_encrypt_sql(connection)
 
-    def get_encrypt_sql(self):
+    def get_encrypt_sql(self, connection):
         """Get encrypt sql. This may be overidden by some implementations."""
         return self.encrypt_sql
 
@@ -92,15 +98,15 @@ def db_type(self, connection=None):
         """Value stored in the database is hexadecimal."""
         return 'bytea'
 
-    def get_placeholder(self, value=None, compiler=None, connection=None):
+    def get_placeholder(self, value, compiler, connection):
         """Tell postgres to encrypt this field using PGP."""
         raise NotImplementedError('The `get_placeholder` needs to be implemented.')
 
     def get_cast_sql(self):
         """Get cast sql. This may be overidden by some implementations."""
         return self.cast_type
 
-    def get_decrypt_sql(self):
+    def get_decrypt_sql(self, connection):
         """Get decrypt sql."""
         raise NotImplementedError('The `get_decrypt_sql` needs to be implemented.')
 
@@ -138,11 +144,11 @@ class PGPPublicKeyFieldMixin(PGPMixin):
 
     def get_placeholder(self, value=None, compiler=None, connection=None):
         """Tell postgres to encrypt this field using PGP."""
-        return self.encrypt_sql.format(settings.PUBLIC_PGP_KEY)
+        return self.encrypt_sql.format(get_setting(connection, 'PUBLIC_PGP_KEY'))
 
-    def get_decrypt_sql(self):
+    def get_decrypt_sql(self, connection):
         """Get decrypt sql."""
-        return self.decrypt_sql.format(settings.PRIVATE_PGP_KEY)
+        return self.decrypt_sql.format(get_setting(connection, 'PRIVATE_PGP_KEY'))
 
 
 class PGPSymmetricKeyFieldMixin(PGPMixin):
@@ -151,13 +157,13 @@ class PGPSymmetricKeyFieldMixin(PGPMixin):
     decrypt_sql = PGP_SYM_DECRYPT_SQL
     cast_type = 'TEXT'
 
-    def get_placeholder(self, value=None, compiler=None, connection=None):
+    def get_placeholder(self, value, compiler, connection):
         """Tell postgres to encrypt this field using PGP."""
-        return self.encrypt_sql.format(settings.PGCRYPTO_KEY)
+        return self.encrypt_sql.format(get_setting(connection, 'PGCRYPTO_KEY'))
 
-    def get_decrypt_sql(self):
+    def get_decrypt_sql(self, connection):
         """Get decrypt sql."""
-        return self.decrypt_sql.format(settings.PGCRYPTO_KEY)
+        return self.decrypt_sql.format(get_setting(connection, 'PGCRYPTO_KEY'))
 
 
 class DecimalPGPFieldMixin:

tests/dbrouters.py

@@ -0,0 +1,13 @@
+class TestRouter(object):
+
+    def db_for_read(self, model, **hints):
+        """Read from diff_keys."""
+        if model._meta.app_label == 'diff_keys':
+            return 'diff_keys'
+        return 'default'
+
+    def db_for_write(self, model, **hints):
+        """Write to diff_keys."""
+        if model._meta.app_label == 'diff_keys':
+            return 'diff_keys'
+        return 'default'

tests/default/__init__.py

@@ -0,0 +1,14 @@
+from django.db import models
+
+from pgcrypto import fields
+
+
+class EncryptedDiff(models.Model):
+    pub_field = fields.TextPGPPublicKeyField()
+    sym_field = fields.TextPGPSymmetricKeyField()
+    digest_field = fields.TextDigestField(blank=True, null=True)
+    hmac_field = fields.TextHMACField(blank=True, null=True)
+
+    class Meta:
+        """Sets up the meta for the test model."""
+        app_label = 'diff_keys'

tests/diff_keys/__init__.py

@@ -0,0 +1,41 @@
+-----BEGIN PGP PRIVATE KEY BLOCK-----
+Version: GnuPG v1
+
+lQNTBFvGJCARCADxYnKHEuntDpQ4URPYhQwBSEDlEJpymFo4ch56KBcEuJCUwRCt
+Ojii7cH9/FMMhvV5ZSx6rGjO+FiHZr+qgjmcCj6PQmLnh2kzj6nF7Q9j1pzlmd84
+/I/93id+7lm0nuWEh317mXfh67pVHb1gagFSSuSPJP4jTja3I8ngn3JV17o630Ae
+P4L/17D4icfW7+BDZ5FH4uWLUR0iYf2LluTXpJPM8tDYNFSvEeD0hqL5IAe9a+x/
+9qujiwTcDWq456m1L4JGvIpxTt7UD0+iwXp1/+FOaPDEiMCQ0VZWO+4RoKnp72Fm
+iKKPS164iSDevYN3Y0BAaESwrrYQvxzGBSwzAQCjWpjfgl1aM1PaqCvtvYBYiNw5
+f1dn8ob3yTv2UUGJ+Qf/f08wKy8UdiJhlX3XIeBPqNi72hHvkeUE/WxXBzygWniG
+sS77Tu+tmFeLMOlV9OxUZqsOswf4pMhK4fPuxTojBGDWQHs7Aso0+fyICUaTF5iG
+kDojXpY3uFuPpKQ0JrujoqS1r3oUDHWON3UMkoS+q9ckP7AqGfovQle8RFxS5Rck
+5/pQFs0z0qRZD4xA9hExjkCCnePxvLh3wiq2VnV1/jBAhmCNn8BmRwXkyDXYcX6k
+1ikpytfmOlTQ2FNc1Pky8Jtkl1vPPimdy5iArKW3pARhnCXEow/LaZp65KzCvluf
+6Tr8+eSxudw3IzeGhKqgbzki52zmaRAGHlwJmXqcSQf/TUvENm57kzeTNWQp1/Ue
+kEirkhNDc1RKTz7RnKi3TaiHKRsE+gvLolaB5uy3NNCIFJN3bjZ5ouXUry06fUa3
+jT4WRecWU2wIsWt1KTd5mAMNfhOZ4I/O9VTgokj9owtRFVmj/jIF10yUmODJIYgm
+F2QMTxuhyCDwMT2VP1Jzn86Qjr3OY6S4DF8zu1V3+D8Es4ZBbUERq8qjyzxICuF0
+5J+cd6I+VYCvba3i3FiRkuLGNS5FHwGUA4upmKK+1DuFeM940539ovveKhNuLYPu
+2SVdKeJPGaiNim/z29B4CRW3a2wiGf7a1brWPJz6T+oy1BE8nTMlmTtGSckNsISa
+9AAA/1hVoJ4xdwN2MvG/aP5HKbN2YGW/1h4sO4YjFDqBzVU9DqK0K0RqYW5nby1Q
+R0NyeXB0by1GaWVsZHMgVGVzdCA8dGVzdEB0ZXN0LmNvbT6IegQTEQgAIgUCW8Yk
+IAIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQ2YBX+jLeYwNfvwEAlZvx
+qApI5jV3BDKqIZhMB3w9X7AkIMndqWY59DhC/2MA/j3ngffgLIon6nKRC92fHtBS
+ypXjL6yIZQWkuufDQg7SnQI9BFvGJCAQCADfFb/TV9sdyE9V028ZIs2EQo4vn6/r
+ZyfycbY3g3cDM1dg4ruDpddcomdXStj7ec3nqlivyl8Abocns77DV137GFXDL1bl
+Eb8QrICwMztkR3e8YTm7y/XTOgJtWVC+XuB45/8zBtfOv30UJIZJT7vHEupLbGCk
+79EuNX2KgIsKBgXrjaE8AP2llEwu98koVOTQFZ3OcNGARfCl/hb9CJGZUNYOTeCQ
+ytX5bO83Vwl1sE3Fi56aTqrstu1yViTHgxavD1zsVCi5fRpwU56bW+g1qxDB5WHi
+Y86zG09DVMzrZ+ORBcEmbva84yoqvV4vPN8QjPiThQBKBJe+IK42iNTjAAMGB/9e
+nRSfHL5+M4OXXoyyj3k4z08Kn4zkMiNsY5dzB8TxzawDNZ03fp6Mh/NQgzAFcszV
+I2E7f+9TSPIkZycyxtQvtjpq4QP/aHjWpxyPVf6T3a8bHb/ZrrBvcYXKL28Co35A
+V2V4ADrqzfB2uHGuXGiiXkFynzgGpdgo6DLWMIDKqKD5BMkFaKzEXKMztr4rRHqr
+5BIMeb+Vsy5lWO+/z6syJNczhRk3gQoQWCEooTqrkqSN3LR9757OHmv+BzIvOoK5
+/PUuz3p7fkjnYsJZ86RN0DJC8hfkh7MY0A/ir3zFgOAKx8M0IpKYy2BL9zxJcesn
+yR9miOHU1NBKgY+nWKGoAAFUDg3hPSO2Be4VUjNKbAgIxA+pf4ufD0lgQMWquKzI
+4roNx4xEw5XL+zZ34BRdiGEEGBEIAAkFAlvGJCACGwwACgkQ2YBX+jLeYwNk+wD+
+LcAaqBxo144tMagsJQ+FgImMW3lRYXJoLz1/cXSPmLIA/iBc1ge/TZOU5h7Lh6hj
+zhkqb4tHw1kFgzXsT9FuzF+r
+=JjTF
+-----END PGP PRIVATE KEY BLOCK-----