Skip to content

Commit 25476da

Browse files
committed
migration 2
1 parent 5bacb64 commit 25476da

File tree

8 files changed

+138
-239
lines changed

8 files changed

+138
-239
lines changed

DynamoDbEncryption/runtimes/python/src/aws_dbesdk_dynamodb/internaldafny/extern/InternalLegacyOverride.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,7 @@ def EncryptItem(self, input):
158158
)
159159

160160
# Use the encryptor to encrypt the item using the instance attributes
161-
encrypted_item = self.encryptor.encrypt_item(
161+
encrypted_item = self.encryptor._encrypt_item(
162162
plaintext_item, actions=self.attribute_actions, encryption_context=self.encryption_context
163163
)
164164

@@ -202,8 +202,8 @@ def DecryptItem(self, input):
202202
)
203203

204204
# Use the encryptor to decrypt the item using the instance attributes
205-
decrypted_item = self.encryptor.decrypt_item(
206-
encrypted_item, actions=self.attribute_actions, encryption_context=self.encryption_context
205+
decrypted_item = self.encryptor._decrypt_item(
206+
encrypted_item,
207207
)
208208

209209
# Create the output with the decrypted item

Examples/runtimes/python/Migration/src/ddbec_to_awsdbe/README.md

Lines changed: 8 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -10,63 +10,46 @@ The migration process is demonstrated through a series of example steps that sho
1010

1111
### Step 0: Legacy DynamoDB Encryption Client
1212

13-
[migration_step0.py](./migration_step0.py) demonstrates using the legacy DynamoDB Encryption Client to encrypt and decrypt items. This represents the starting point for migration.
13+
[migration_step_0.py](./ddbec/migration_step_0.py) demonstrates using the legacy DynamoDB Encryption Client to encrypt and decrypt items. This represents the starting point for migration.
1414

1515
Key concepts:
16+
1617
- Setting up the legacy client with an AWS KMS cryptographic materials provider
1718
- Defining attribute actions for encryption/signing
1819
- Storing and retrieving encrypted items
1920

2021
### Step 1: AWS Database Encryption SDK with Legacy Override
2122

22-
[migration_step1.py](./migration_step1.py) demonstrates how to start using the AWS Database Encryption SDK with a pre-existing table used with the DynamoDB Encryption Client.
23+
[migration_step_1.py](./awsdbe/migration_step_1.py) demonstrates how to start using the AWS Database Encryption SDK with a pre-existing table used with the DynamoDB Encryption Client.
2324

2425
Key concepts:
26+
2527
- Configure AWS DBESDK to read items encrypted in the legacy format
2628
- Continue to encrypt items in the legacy format (FORCE_LEGACY_ENCRYPT_ALLOW_DECRYPT policy)
2729
- Read items encrypted in the new format
2830
- Deploy this step to all readers before moving to step 2
2931

3032
### Step 2: Full Migration to AWS Database Encryption SDK
3133

32-
[migration_step2.py](./migration_step2.py) demonstrates the next step in the migration process, using both the pure AWS DBESDK client and the legacy-override client side by side.
34+
[migration_step_2.py](./awsdbe/migration_step_2.py) demonstrates the next step in the migration process, using both the pure AWS DBESDK client and the legacy-override client side by side.
3335

3436
Key concepts:
37+
3538
- Create a pure AWS DBESDK client for new data
3639
- Keep using legacy-override client when needed for legacy data
3740
- Re-encrypt legacy data with the new client
3841
- Demonstrate that the legacy-override client can read both formats
3942

4043
### Step 3: Complete Migration - Using Only AWS DBESDK
4144

42-
[migration_step3.py](./migration_step3.py) demonstrates the final state of the migration, where all data has been re-encrypted using the new format.
45+
[migration_step_3.py](./awsdbe/migration_step_3.py) demonstrates the final state of the migration, where all data has been re-encrypted using the new format.
4346

4447
Key concepts:
48+
4549
- Use only the pure AWS DBESDK client (no more legacy override)
4650
- Verify all previously re-encrypted data is readable
4751
- Add new data using the pure client
4852

49-
### Run All Migration Steps
50-
51-
[run_all.py](./run_all.py) runs all four migration steps in sequence, demonstrating the entire migration path.
52-
53-
## How to Run
54-
55-
To run an individual step:
56-
57-
```bash
58-
python -m Examples.runtimes.python.DynamoDBEncryption.src.migration.ddbec_to_awsdbe.migration_step0
59-
python -m Examples.runtimes.python.DynamoDBEncryption.src.migration.ddbec_to_awsdbe.migration_step1
60-
python -m Examples.runtimes.python.DynamoDBEncryption.src.migration.ddbec_to_awsdbe.migration_step2
61-
python -m Examples.runtimes.python.DynamoDBEncryption.src.migration.ddbec_to_awsdbe.migration_step3
62-
```
63-
64-
To run all steps in sequence:
65-
66-
```bash
67-
python -m Examples.runtimes.python.DynamoDBEncryption.src.migration.ddbec_to_awsdbe.run_all
68-
```
69-
7053
## Prerequisites
7154

7255
Before running these examples:

Examples/runtimes/python/Migration/src/ddbec_to_awsdbe/awsdbe/common.py

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@ def setup_pure_awsdbe_client(kms_key_id: str, ddb_table_name: str):
129129
)
130130

131131

132-
def setup_awsdbe_client_with_legacy_override(kms_key_id: str, ddb_table_name: str):
132+
def setup_awsdbe_client_with_legacy_override(kms_key_id: str, ddb_table_name: str, policy: str):
133133
"""
134134
Set up a pure AWS Database Encryption SDK client without legacy override.
135135
@@ -151,14 +151,12 @@ def setup_awsdbe_client_with_legacy_override(kms_key_id: str, ddb_table_name: st
151151
materials_provider=cmp,
152152
)
153153

154-
# 2. Configure our legacy behavior, inputting the DynamoDBEncryptor and attribute actions
155-
# created above. For Legacy Policy, use `FORCE_LEGACY_ENCRYPT_ALLOW_LEGACY_DECRYPT`.
156-
# With this policy, you will continue to read and write items using the old format,
157-
# but will be able to start reading new items in the new format as soon as they appear.
154+
# 2. Configure our legacy behavior, inputting the DynamoDBEncryptor, attribute actions
155+
# created above, and legacy pooicy.
158156
legacy_override = LegacyOverride(
159157
encryptor=legacy_encrypted_client,
160158
attribute_actions_on_encrypt=ATTRIBUTE_ACTIONS_ON_ENCRYPT,
161-
policy=LegacyPolicy.FORCE_LEGACY_ENCRYPT_ALLOW_LEGACY_DECRYPT,
159+
policy=policy,
162160
)
163161

164162
# 1. Create a Keyring. This Keyring will be responsible for protecting the data keys that protect your data.

Examples/runtimes/python/Migration/src/ddbec_to_awsdbe/awsdbe/migration_step_1.py

Lines changed: 19 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
- Partition key is named "partition_key" with type (S)
2323
- Sort key is named "sort_key" with type (S)
2424
"""
25+
from aws_dbesdk_dynamodb.structures.dynamodb import LegacyPolicy
2526

2627
import boto3
2728

@@ -30,7 +31,7 @@
3031
from .common import setup_awsdbe_client_with_legacy_override
3132

3233

33-
def migration_step_1(kms_key_id, ddb_table_name, sort_read_value=0):
34+
def migration_step_1(kms_key_id: str, ddb_table_name: str, sort_read_value: int=1):
3435
"""
3536
Migration Step 1: Using the AWS Database Encryption SDK with Legacy Override.
3637
@@ -40,13 +41,14 @@ def migration_step_1(kms_key_id, ddb_table_name, sort_read_value=0):
4041
sort_read_value: The sort key value to read
4142
4243
"""
43-
# 1. Create AWS SDK DynamoDB Client
44-
ddb_client = boto3.client("dynamodb")
45-
46-
# 2. Create a DynamoDB Encryption SDK client with legacy override
47-
encrypted_client = setup_awsdbe_client_with_legacy_override(kms_key_id=kms_key_id, ddb_table_name=ddb_table_name)
48-
49-
# 3. Put an item in the old format since we are using a legacy override
44+
# 1. Create a DynamoDB Encryption SDK client with legacy override.
45+
# For Legacy Policy, use `FORCE_LEGACY_ENCRYPT_ALLOW_LEGACY_DECRYPT`.
46+
# With this policy, you will continue to read and write items using the old format,
47+
# but will be able to start reading new items in the new format as soon as they appear
48+
policy=LegacyPolicy.FORCE_LEGACY_ENCRYPT_ALLOW_LEGACY_DECRYPT
49+
encrypted_client = setup_awsdbe_client_with_legacy_override(kms_key_id=kms_key_id, ddb_table_name=ddb_table_name, policy=policy)
50+
51+
# 2. Put an item in the old format since we are using a legacy override
5052
# with FORCE_LEGACY_ENCRYPT_ALLOW_DECRYPT policy
5153
item_to_encrypt = {
5254
"partition_key": {"S": "MigrationExampleForPython"},
@@ -67,24 +69,24 @@ def migration_step_1(kms_key_id, ddb_table_name, sort_read_value=0):
6769
print("Put Item Response SHOULD have legacy attributes")
6870
print(f"Put Item Response: {put_item_response['Item']}")
6971

70-
# 4. Get the item back from DynamoDB directly and decrypt it with our client.
71-
# Because we used FORCE_LEGACY_ENCRYPT_ALLOW_DECRYPT, we expect the item to
72-
# be encrypted in the old format.
72+
# 3. Get an item back from the table using the DynamoDb Enhanced Client.
73+
# If this is an item written in the old format (e.g. any item written
74+
# during Step 0 or 1), then we will attempt to decrypt the item
75+
# using the legacy behavior.
76+
# If this is an item written in the new format (e.g. any item written
77+
# during Step 2 or after), then we will attempt to decrypt the item using
78+
# the non-legacy behavior.
7379
key_to_get = {"partition_key": {"S": "MigrationExampleForPython"}, "sort_key": {"N": str(sort_read_value)}}
7480

7581
get_item_request = {"TableName": ddb_table_name, "Key": key_to_get}
7682
get_item_response = encrypted_client.get_item(**get_item_request)
7783

7884
# Demonstrate that GetItem succeeded and returned the decrypted item
7985
assert get_item_response["ResponseMetadata"]["HTTPStatusCode"] == 200
80-
assert get_item_response["Item"] == item_to_encrypt
81-
8286
decrypted_item = get_item_response["Item"]
8387
# Demonstrate we get the expected item back
8488
assert decrypted_item["partition_key"]["S"] == "MigrationExampleForPython"
85-
if sort_read_value == 0:
86-
assert decrypted_item["attribute1"]["S"] == "encrypt and sign me!"
87-
elif sort_read_value == 1:
88-
assert decrypted_item["attribute1"]["S"] == "encrypt and sign me!"
89+
assert decrypted_item["sort_key"]["N"] == str(sort_read_value)
90+
assert decrypted_item["attribute1"]["S"] == "encrypt and sign me!"
8991
assert decrypted_item["attribute2"]["S"] == "sign me!"
9092
assert decrypted_item["attribute3"]["S"] == "ignore me!"

Examples/runtimes/python/Migration/src/ddbec_to_awsdbe/awsdbe/migration_step_2.py

Lines changed: 45 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -25,130 +25,69 @@
2525
"""
2626

2727
import boto3
28+
from aws_dbesdk_dynamodb.structures.dynamodb import LegacyPolicy
2829

2930
# Import from new AWS Database Encryption SDK
3031
from .common import setup_awsdbe_client_with_legacy_override, setup_pure_awsdbe_client
3132

3233

33-
def migration_step_2(kms_key_id, ddb_table_name):
34+
def migration_step_2(kms_key_id: str, ddb_table_name: str, sort_read_value: int=2):
3435
"""
3536
Migration Step 2: Using pure AWS DBESDK and legacy override together.
3637
3738
Args:
3839
kms_key_id: The ARN of the KMS key to use for encryption
3940
ddb_table_name: The name of the DynamoDB table
41+
sort_read_value: The sort key value to read
4042
4143
"""
42-
ddb_client = boto3.client("dynamodb")
43-
44-
# 1. Create two AWS DBESDK clients:
45-
# - A pure client that can only read items encrypted with AWS DBESDK
46-
# - A client with legacy override that can read legacy items
47-
pure_client = setup_pure_awsdbe_client(kms_key_id, ddb_table_name)
48-
legacy_override_client = setup_awsdbe_client_with_legacy_override(kms_key_id, ddb_table_name)
49-
50-
# 2. Write a new item using the pure client (it will use the new encryption format)
51-
item = {
44+
# 1. Create a DynamoDB Encryption SDK client with legacy override.
45+
# When configuring our legacy behavior, use `FORBID_LEGACY_ENCRYPT_ALLOW_LEGACY_DECRYPT`.
46+
# With this policy, you will continue to read items in both formats,
47+
# but will only write new items using the new format.
48+
encrypted_client = setup_awsdbe_client_with_legacy_override(kms_key_id, ddb_table_name, policy=LegacyPolicy.FORBID_LEGACY_ENCRYPT_ALLOW_LEGACY_DECRYPT)
49+
50+
# 2. Put an item into your table using the DB ESDK Client.
51+
# This item will be encrypted in the latest format, using the
52+
# configuration from your modelled class to decide
53+
# which attribute to encrypt and/or sign.
54+
item_to_encrypt = {
5255
"partition_key": {"S": "MigrationExampleForPython"},
5356
"sort_key": {"N": str(2)},
5457
"attribute1": {"S": "encrypt and sign me!"},
5558
"attribute2": {"S": "sign me!"},
5659
"attribute3": {"S": "ignore me!"},
5760
}
5861

59-
encrypted_item = pure_client.EncryptItem(item)
60-
ddb_client.put_item(TableName=ddb_table_name, Item=encrypted_item)
61-
print("Put item with sort_key=2 using pure AWS DBESDK client")
62-
63-
# 3. Attempt to read a legacy-encrypted item (sort_key=0) with the pure client
64-
# This should fail as the pure client doesn't understand the legacy format
65-
key = {"partition_key": {"S": "MigrationExampleForPython"}, "sort_key": {"N": str(0)}}
66-
67-
response = ddb_client.get_item(TableName=ddb_table_name, Key=key)
68-
69-
if "Item" in response:
70-
legacy_item = response["Item"]
71-
try:
72-
pure_client.DecryptItem(legacy_item)
73-
print("Unexpectedly succeeded in decrypting legacy item with pure client!")
74-
except Exception as e:
75-
print(f"Expected failure: Pure client can't decrypt legacy item: {type(e).__name__}")
76-
77-
# 4. Read the same legacy item using the client with legacy override
78-
# This should succeed as this client understands both legacy and new formats
79-
if "Item" in response:
80-
legacy_item = response["Item"]
81-
legacy_override_client.DecryptItem(legacy_item)
82-
print("Successfully decrypted legacy item (sort_key=0) using client with legacy override")
83-
84-
# 5. Read the new item (sort_key=2) with both clients
85-
# Both should be able to read the new format
86-
key = {"partition_key": {"S": "MigrationExampleForPython"}, "sort_key": {"N": str(2)}}
87-
88-
response = ddb_client.get_item(TableName=ddb_table_name, Key=key)
89-
90-
if "Item" in response:
91-
new_item = response["Item"]
92-
pure_client.DecryptItem(new_item)
93-
print("Successfully decrypted new item (sort_key=2) using pure client")
94-
95-
legacy_override_client.DecryptItem(new_item)
96-
print("Successfully decrypted new item (sort_key=2) using client with legacy override")
97-
98-
# 6. Re-encrypt a legacy item with the new format
99-
# This is how you migrate your data from the legacy format to the new format
100-
key = {"partition_key": {"S": "MigrationExampleForPython"}, "sort_key": {"N": str(0)}}
101-
102-
response = ddb_client.get_item(TableName=ddb_table_name, Key=key)
103-
104-
if "Item" in response:
105-
# Get the legacy encrypted item
106-
legacy_item = response["Item"]
107-
108-
# Decrypt it with the client that has legacy override
109-
decrypted_item = legacy_override_client.DecryptItem(legacy_item)
110-
print("Successfully decrypted legacy item for re-encryption")
111-
112-
# Re-encrypt with the pure client (new format)
113-
new_encrypted_item = pure_client.EncryptItem(decrypted_item)
114-
print("Successfully re-encrypted item in new format")
115-
116-
# Store the re-encrypted item back in DynamoDB
117-
ddb_client.put_item(TableName=ddb_table_name, Item=new_encrypted_item)
118-
print("Successfully stored re-encrypted item")
119-
120-
# Verify we can now read it with the pure client
121-
verify_response = ddb_client.get_item(TableName=ddb_table_name, Key=key)
122-
123-
if "Item" in verify_response:
124-
re_encrypted_item = verify_response["Item"]
125-
pure_client.DecryptItem(re_encrypted_item)
126-
print("Successfully verified re-encrypted item can be read with pure client")
127-
128-
129-
# def run_example():
130-
# """Run the full migration example."""
131-
132-
# print("\n=== Migration Example Step 2: Using Pure AWS DBESDK and Legacy Override Together ===")
133-
134-
# try:
135-
# # Ensure we have items from previous steps
136-
# # First run migration step 0 to have legacy items
137-
# migration_step0.migration_step0(
138-
# kms_key_id=common.KMS_KEY_ID,
139-
# ddb_table_name=common.TABLE_NAME
140-
# )
141-
142-
# # Now run migration step 2
143-
# migration_step2(
144-
# kms_key_id=common.KMS_KEY_ID,
145-
# ddb_table_name=common.TABLE_NAME
146-
# )
147-
148-
# print("\nMigration Step 2 completed successfully!")
149-
# print("This demonstrates how to use both clients side-by-side and")
150-
# print("how to re-encrypt legacy data to the new format.")
62+
put_item_request = {
63+
"TableName": ddb_table_name,
64+
"Item": item_to_encrypt,
65+
}
15166

152-
# except Exception as e:
153-
# print(f"Error in Migration Step 2: {e}")
154-
# raise
67+
put_item_response = encrypted_client.put_item(**put_item_request)
68+
# Demonstrate that PutItem succeeded
69+
assert put_item_response["ResponseMetadata"]["HTTPStatusCode"] == 200
70+
print("Put Item Response SHOULD have legacy attributes")
71+
print(f"Put Item Response: {put_item_response['Item']}")
72+
73+
# 3. Get an item back from the table using the Client.
74+
# If this is an item written in the old format (e.g. any item written
75+
# during Step 0 or 1), then we will attempt to decrypt the item
76+
# using the legacy behavior.
77+
# If this is an item written in the new format (e.g. any item written
78+
# during Step 2 or after), then we will attempt to decrypt the item using
79+
# the non-legacy behavior.
80+
key_to_get = {"partition_key": {"S": "MigrationExampleForPython"}, "sort_key": {"N": str(sort_read_value)}}
81+
82+
get_item_request = {"TableName": ddb_table_name, "Key": key_to_get}
83+
get_item_response = encrypted_client.get_item(**get_item_request)
84+
85+
# Demonstrate that GetItem succeeded and returned the decrypted item
86+
assert get_item_response["ResponseMetadata"]["HTTPStatusCode"] == 200
87+
decrypted_item = get_item_response["Item"]
88+
# Demonstrate we get the expected item back
89+
assert decrypted_item["partition_key"]["S"] == "MigrationExampleForPython"
90+
assert decrypted_item["sort_key"]["N"] == str(sort_read_value)
91+
assert decrypted_item["attribute1"]["S"] == "encrypt and sign me!"
92+
assert decrypted_item["attribute2"]["S"] == "sign me!"
93+
assert decrypted_item["attribute3"]["S"] == "ignore me!"

0 commit comments

Comments
 (0)