feat(python): add state machine to collect metrics for dashboard

* update the hard-coded Python-Version for the Lambda Function
* Lambda Function code is now inline
* uv-ready with pyproject.toml
* new Stack: StepFunction state machine to automatically collect metrics for the dashboard
* python code ruff formated

Author: ronfoerster

python/lambda-cloudwatch-dashboard/README.md

@@ -7,49 +7,66 @@ CloudWatch dashboards are used to create customized views of the metrics and ala
 This CDK sample uses an AWS Lambda Function, as an example, for the source of CloudWatch metrics. This
 approach can used with AWS Services that create [CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/working_with_metrics.html) or even [Custom CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html) that you publish yourself.
 
-The following resources are defined in the CDK Stack:
-- [AWS Lambda Function](https://aws.amazon.com/lambda/)
+The following resources are defined in the main CDK Stack:
 - [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/)
+- [AWS Lambda Function](https://aws.amazon.com/lambda/)
+
+To automatically trigger the Lambda Function we use an
+[AWS StepFunctions](https://aws.amazon.com/step-functions/), started after provisioning by a Custom Resource(another Lambda Function).
 
-After deploying solution, you will have created a CloudWatch Dashboard, like the one shown below:
+After deploying the CDK Stacks the StepFunctions state machine will run 20+ minutes to generate sample metrics.
+
+In CloudWatch you see the provisioned dashboard, like the one shown below:
 
 ![Sample Dashboard](img/sample_cloudwatch_dashboard.png)
 
----
-### Requirements:
+## Setup
+
+This project is set up like a standard Python project.
 
-- git
-- npm (node.js)
-- python 3.x
-- AWS access key & secret for AWS user with permissions to create resources listed above
+## Use uv
+Create a virtualenv and install all dependencies
+```
+uv sync
+```
+At this point you can now synthesize the CloudFormation template for this code.
+```
+uv run cdk synth
+```
+Or simple proceed to deployment of the stack.
+```
+uv run cdk deploy --all
+```
 
----
+### View CloudWatch Dashboard
 
-## Setup
+1) Sign in to the AWS Console
+2) Navigate to the URL in this CDK Stack Output: `LambdaCloudwatchDashboardStack.DashboardOutput`
+3) Please note, the metrics are aggregated for a period of 5 minutes before being displayed on the Dashboard.  The value of the period can be configured, please see the [CDK documentation](https://docs.aws.amazon.com/cdk/api/latest/python/aws_cdk.aws_cloudwatch/MetricProps.html) for further details.
 
-This project is set up like a standard Python project.  The initialization process also creates
-a virtualenv within this project, stored under the .env directory.  To create the virtualenv
-it assumes that there is a `python3` executable in your path with access to the `venv` package.
-If for any reason the automatic creation of the virtualenv fails, you can create the virtualenv
-manually once the init process completes.
+To clean up, issue this command:
+```
+uv run cdk destroy --all
+```
 
+## Or use the default deployment
 To manually create a virtualenv on MacOS and Linux:
 
 ```
-$ python3 -m venv .env
+$ python3 -m venv .venv
 ```
 
 After the init process completes and the virtualenv is created, you can use the following
 step to activate your virtualenv.
 
 ```
-$ source .env/bin/activate
+$ source .venv/bin/activate
 ```
 
 If you are a Windows platform, you would activate the virtualenv like this:
 
 ```
-$ .env\Scripts\activate.bat
+% .venv\Scripts\activate.bat
 ```
 
 Once the virtualenv is activated, you can install the required dependencies.
@@ -58,15 +75,6 @@ Once the virtualenv is activated, you can install the required dependencies.
 $ pip install -r requirements.txt
 ```
 
-
-Install the latest version of the AWS CDK CLI:
-
-```
-$ npm i -g aws-cdk
-```
-
-## Deployment
-
 At this point you can now synthesize the CloudFormation template for this code.
 ```
 cdk synth
@@ -76,36 +84,8 @@ Or simple proceed to deployment of the stack.
 ```
 cdk deploy
 ```
-
-## Test
-
-### Invoke Lambda Function
-In order to generate some metrics, you can invoke the sample Lambda Function:
-
-Replace `<NAME_OF_FUNCTION>` with the value of this CDK Stack Output: `LambdaCloudwatchDashboardStack.LambdaName`
-```
-aws lambda invoke --function-name <NAME_OF_FUNCTION> text_output.txt
-```
-
-### View CloudWatch Dashboard
-
-1) Sign into to the AWS Console
-2) Navigate to the URL in this CDK Stack Output: `LambdaCloudwatchDashboardStack.DashboardOutput`
-3) Please note, the metrics are aggregated for a period of 5 minutes before being displayed on the Dashboard.  The value of the period can be configured, please see the [CDK documentation](https://docs.aws.amazon.com/cdk/api/latest/python/aws_cdk.aws_cloudwatch/MetricProps.html) for further details.
-
-
 ## Clean Up
 To clean up, issue this command:
 ```
 cdk destroy
-```
-
-## Useful commands
-
- * `cdk ls`          list all stacks in the app
- * `cdk synth`       emits the synthesized CloudFormation template
- * `cdk deploy`      deploy this stack to your default AWS account/region
- * `cdk diff`        compare deployed stack with current state
- * `cdk docs`        open CDK documentation
-
-Enjoy!
+```

python/lambda-cloudwatch-dashboard/app.py

@@ -1,8 +1,19 @@
 #!/usr/bin/env python3
+
 from aws_cdk import App
-from lambda_cloudwatch_dashboard.lambda_cloudwatch_dashboard_stack import LambdaCloudwatchDashboardStack
 
+from lambda_cloudwatch_dashboard.lambda_cloudwatch_dashboard_stack import (
+    LambdaCloudwatchDashboardStack,
+)
+from lambda_cloudwatch_dashboard.stepfunctions_invoker_stack import (
+    StepfunctionsInvokerStack,
+)
 
 app = App()
-LambdaCloudwatchDashboardStack(app, "LambdaCloudwatchDashboardStack")
+
+lambdastack = LambdaCloudwatchDashboardStack(app, "LambdaCloudwatchDashboardStack")
+
+stepfunctionsstack = StepfunctionsInvokerStack(app, "StepfunctionsInvokerStack")
+stepfunctionsstack.add_dependency(lambdastack)
+
 app.synth()

python/lambda-cloudwatch-dashboard/cdk.json

@@ -1,3 +1,117 @@
 {
-  "app": "python3 app.py"
+  "app": "python3 app.py",
+  "staging": "false",
+  "watch": {
+    "include": [
+      "**"
+    ],
+    "exclude": [
+      "README.md",
+      "cdk*.json",
+      "pyproject.toml",
+      "requirements*.txt",
+      "source.bat",
+      "**/__init__.py",
+      "**/__pycache__",
+      "tests"
+    ]
+  },
+  "context": {
+    "@aws-cdk/aws-signer:signingProfileNamePassedToCfn": true,
+    "@aws-cdk/aws-ecs-patterns:secGroupsDisablesImplicitOpenListener": true,
+    "@aws-cdk/aws-lambda:recognizeLayerVersion": true,
+    "@aws-cdk/core:checkSecretUsage": true,
+    "@aws-cdk/core:target-partitions": [
+      "aws",
+      "aws-cn"
+    ],
+    "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
+    "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
+    "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
+    "@aws-cdk/aws-iam:minimizePolicies": true,
+    "@aws-cdk/core:validateSnapshotRemovalPolicy": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
+    "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
+    "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
+    "@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
+    "@aws-cdk/core:enablePartitionLiterals": true,
+    "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
+    "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
+    "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
+    "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
+    "@aws-cdk/aws-route53-patters:useCertificate": true,
+    "@aws-cdk/customresources:installLatestAwsSdkDefault": false,
+    "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
+    "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
+    "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
+    "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
+    "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
+    "@aws-cdk/aws-redshift:columnId": true,
+    "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
+    "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
+    "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
+    "@aws-cdk/aws-kms:aliasNameRef": true,
+    "@aws-cdk/aws-kms:applyImportedAliasPermissionsToPrincipal": true,
+    "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
+    "@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
+    "@aws-cdk/aws-efs:denyAnonymousAccess": true,
+    "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
+    "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
+    "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
+    "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
+    "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true,
+    "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true,
+    "@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true,
+    "@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true,
+    "@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true,
+    "@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true,
+    "@aws-cdk/aws-eks:nodegroupNameAttribute": true,
+    "@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true,
+    "@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true,
+    "@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false,
+    "@aws-cdk/aws-s3:keepNotificationInImportedBucket": false,
+    "@aws-cdk/core:explicitStackTags": true,
+    "@aws-cdk/aws-ecs:enableImdsBlockingDeprecatedFeature": false,
+    "@aws-cdk/aws-ecs:disableEcsImdsBlocking": true,
+    "@aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions": true,
+    "@aws-cdk/aws-dynamodb:resourcePolicyPerReplica": true,
+    "@aws-cdk/aws-ec2:ec2SumTImeoutEnabled": true,
+    "@aws-cdk/aws-appsync:appSyncGraphQLAPIScopeLambdaPermission": true,
+    "@aws-cdk/aws-rds:setCorrectValueForDatabaseInstanceReadReplicaInstanceResourceId": true,
+    "@aws-cdk/core:cfnIncludeRejectComplexResourceUpdateCreatePolicyIntrinsics": true,
+    "@aws-cdk/aws-lambda-nodejs:sdkV3ExcludeSmithyPackages": true,
+    "@aws-cdk/aws-stepfunctions-tasks:fixRunEcsTaskPolicy": true,
+    "@aws-cdk/aws-ec2:bastionHostUseAmazonLinux2023ByDefault": true,
+    "@aws-cdk/aws-route53-targets:userPoolDomainNameMethodWithoutCustomResource": true,
+    "@aws-cdk/aws-elasticloadbalancingV2:albDualstackWithoutPublicIpv4SecurityGroupRulesDefault": true,
+    "@aws-cdk/aws-iam:oidcRejectUnauthorizedConnections": true,
+    "@aws-cdk/core:enableAdditionalMetadataCollection": true,
+    "@aws-cdk/aws-lambda:createNewPoliciesWithAddToRolePolicy": false,
+    "@aws-cdk/aws-s3:setUniqueReplicationRoleName": true,
+    "@aws-cdk/aws-events:requireEventBusPolicySid": true,
+    "@aws-cdk/core:aspectPrioritiesMutating": true,
+    "@aws-cdk/aws-dynamodb:retainTableReplica": true,
+    "@aws-cdk/aws-stepfunctions:useDistributedMapResultWriterV2": true,
+    "@aws-cdk/s3-notifications:addS3TrustKeyPolicyForSnsSubscriptions": true,
+    "@aws-cdk/aws-ec2:requirePrivateSubnetsForEgressOnlyInternetGateway": true,
+    "@aws-cdk/aws-s3:publicAccessBlockedByDefault": true,
+    "@aws-cdk/aws-lambda:useCdkManagedLogGroup": true,
+    "@aws-cdk/core:newStyleStackSynthesis": true,
+    "@aws-cdk/core:stackRelativeExports": true,
+    "@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
+    "@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
+    "@aws-cdk/aws-lambda:recognizeVersionProps": true,
+    "@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true,
+    "@aws-cdk/pipelines:reduceAssetRoleTrustScope": true,
+    "@aws-cdk/aws-stepfunctions-tasks:useNewS3UriParametersForBedrockInvokeModelTask": true,
+    "@aws-cdk/core:aspectStabilization": true,
+    "@aws-cdk/pipelines:reduceStageRoleTrustScope": true,
+    "@aws-cdk/cognito:logUserPoolClientSecretValue": false,
+    "@aws-cdk/pipelines:reduceCrossAccountActionRoleTrustScope": true,
+    "@aws-cdk/aws-ec2-alpha:useResourceIdForVpcV2Migration": false,
+    "@aws-cdk/aws-elasticloadbalancingv2:networkLoadBalancerWithSecurityGroupByDefault": true,
+    "@aws-cdk/aws-stepfunctions-tasks:httpInvokeDynamicJsonPathEndpoint": true,
+    "@aws-cdk/aws-ecs-patterns:uniqueTargetGroupId": true
+  }
 }