Merge pull request #9 from aws-solutions/feature/v1.1.0

Release candidate/v1.1.0

Author: dscpinheiro

.github/ISSUE_TEMPLATE/bug_report.md

@@ -17,7 +17,7 @@ Steps to reproduce the behavior.
 A clear and concise description of what you expected to happen.
 
 **Please complete the following information about the solution:**
-- [ ] Version: [e.g. v1.0.0]
+- [ ] Version: [e.g. v0.0.1]
 
 To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "(SO0170) Maintaining Personalized Experiences with Machine Learning [...]".
 

.gitignore

@@ -48,9 +48,9 @@ __pycache__/
 # Generated test assets
 source/infrastructure/tests/assets/*
 !source/infrastructure/tests/assets/.keep
-source/aws_lambda/get_next_scheduled_event/.gradle
-source/aws_lambda/get_next_scheduled_event/build
-source/aws_lambda/get_next_scheduled_event/.idea
+source/scheduler/cdk/aws_solutions/scheduler/cdk/aws_lambda/get_next_scheduled_event/build
+source/scheduler/cdk/aws_solutions/scheduler/cdk/aws_lambda/get_next_scheduled_event/.gradle
+source/scheduler/cdk/aws_solutions/scheduler/cdk/aws_lambda/get_next_scheduled_event/.idea
 
 # gradle build files
 **/.gradle/*
@@ -60,4 +60,6 @@ source/aws_lambda/get_next_scheduled_event/.idea
 
 # python build files
 source/cdk_solution_helper_py/helpers_cdk/build/*
-source/cdk_solution_helper_py/helpers_common/build/*
+source/cdk_solution_helper_py/helpers_common/build/*
+source/scheduler/common/build/*
+source/scheduler/cdk/build/*

CHANGELOG.md

@@ -4,6 +4,13 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.1.0] - 2021-11-22
+### Added
+- The solution now creates an Amazon EventBridge event bus, and puts messages to the bus when resources have been
+created by the workflow. This can be useful when integrating with external systems. 
+- The solution now contains a command line interface (CLI) that allows schedule creation for existing resources in
+Amazon Personalize. 
+
 ## [1.0.1] - 2021-10-01
 ### Added
 - The solution now exports the Amazon SNS Topic ARN as `SNSTopicArn`.

NOTICE.txt

@@ -34,6 +34,7 @@ pytest-env under the Massachusetts Institute of Technology (MIT) license
 PyYAML under the Massachusetts Institute of Technology (MIT) license
 requests under the Apache License Version 2.0
 requests-mock under the Apache License Version 2.0
+rich under the Massachusetts Institute of Technology (MIT) license
 tenacity under the Apache License Version 2.0
 quartz-scheduler under the Apache License Version 2.0
 

README.md

@@ -35,6 +35,10 @@ The template includes the following components:
     2. Perform solution FULL retraining on schedule (and update associated campaigns)
     3. Perform solution UPDATE retraining on schedule (and update associated campaigns)
     4. Create batch inference jobs
+9. An Amazon EventBridge event bus, where resource status notification updates are posted throughout the AWS Step
+functions workflow 
+10. A command line interface (CLI) lets existing resources be imported and allows schedules to be established for
+resources that already exist in Amazon Personalize
 
 
 **Note**: From v1.0.0, AWS CloudFormation template resources are created by the [AWS CDK](https://aws.amazon.com/cdk/) 
@@ -287,9 +291,9 @@ To customize the solution, follow the steps below:
 The following procedures assumes that all the OS-level configuration has been completed. They are:
 
 * [AWS Command Line Interface](https://aws.amazon.com/cli/)
-* [Python](https://www.python.org/) 3.7 or newer
+* [Python](https://www.python.org/) 3.9 or newer
 * [Node.js](https://nodejs.org/en/) 16.x or newer 
-* [AWS CDK](https://aws.amazon.com/cdk/) 1.95.2 or newer 
+* [AWS CDK](https://aws.amazon.com/cdk/) 1.126.0 or newer 
 * [Amazon Corretto OpenJDK](https://docs.aws.amazon.com/corretto/) 11 
 
 > **Please ensure you test the templates before updating any production deployments.**
@@ -360,7 +364,7 @@ build-s3-cdk-dist \
   S3 bucket where the name is `<DIST_BUCKET_PREFIX>-<REGION_NAME>`. The solution's CloudFormation template will expect the
   source code to be located in the bucket matching that name.
 - `$SOLUTION_NAME` - The name of This solution (example: personalize-solution-customization)
-- `$VERSION` - The version number to use (example: v1.0.1)
+- `$VERSION` - The version number to use (example: v0.0.1)
 - `$REGION_NAME` - The region name to use (example: us-east-1)
 
 This will result in all global assets being pushed to the `DIST_BUCKET_PREFIX`, and all regional assets being pushed to 

source/.coveragerc

@@ -7,6 +7,7 @@ source =
     infrastructure
     aws_lambda
     cdk_solution_helper_py
+    scheduler
 
 [report]
 fail_under = 80.0

source/aws_lambda/create_batch_inference_job/handler.py

@@ -64,6 +64,12 @@
             "default": "omit",
             "as": "seconds",
         },
+        "timeStarted": {
+            "source": "event",
+            "path": "workflowConfig.timeStarted",
+            "default": "omit",
+            "as": "iso8601",
+        },
     },
 )
 def lambda_handler(event: Dict[str, Any], context: LambdaContext) -> Dict:

source/aws_lambda/create_campaign/handler.py

@@ -52,8 +52,14 @@
             "default": "omit",
             "as": "seconds",
         },
+        "timeStarted": {
+            "source": "event",
+            "path": "workflowConfig.timeStarted",
+            "default": "omit",
+            "as": "iso8601",
+        },
     },
-    status="campaign.status",
+    status="campaign.latestCampaignUpdate.status || campaign.status",
 )
 def lambda_handler(event: Dict[str, Any], context: LambdaContext) -> Dict:
     """Create a campaign in Amazon Personalize based on the configuration in `event`

source/aws_lambda/scheduler/__init__.py renamed to source/aws_lambda/create_config/__init__.py

@@ -0,0 +1,42 @@
+# ######################################################################################################################
+#  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.                                                  #
+#                                                                                                                      #
+#  Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance      #
+#  with the License. You may obtain a copy of the License at                                                           #
+#                                                                                                                      #
+#   http://www.apache.org/licenses/LICENSE-2.0                                                                         #
+#                                                                                                                      #
+#  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed    #
+#  on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for   #
+#  the specific language governing permissions and limitations under the License.                                      #
+# ######################################################################################################################
+import json
+import os
+
+from aws_lambda_powertools import Logger, Tracer, Metrics
+from aws_lambda_powertools.metrics import MetricUnit
+from aws_lambda_powertools.utilities.data_classes import S3Event
+
+from shared.personalize.service_model import ServiceModel
+from shared.personalize_service import Personalize
+
+
+logger = Logger()
+tracer = Tracer()
+metrics = Metrics()
+
+
+@metrics.log_metrics
+@tracer.capture_lambda_handler
+def lambda_handler(event, context):
+    """Generate and return a solution configuration file derived from the properties of a dataset group
+    :param dict event: AWS Lambda Event (in this case, the dataset group and schedules)
+    :param context: AWS Lambda Context
+    :return: Dict
+    """
+    dataset_group_name = event["datasetGroupName"]
+    schedules = event.get("schedules")
+
+    cli = Personalize()
+    model = ServiceModel(cli, dataset_group_name=dataset_group_name)
+    return model.get_config(dataset_group_name=dataset_group_name, schedules=schedules)