Skip to content

Commit f3150f5

Browse files
authored
Merge pull request #2510 from philippewanner/philippewanner-feature-iot-rule-action-lambda-python-cdk-ts
Philippewanner feature iot rule action lambda python cdk ts
2 parents a4508b8 + 6e45f48 commit f3150f5

File tree

12 files changed

+523
-0
lines changed

12 files changed

+523
-0
lines changed
Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# AWS Lambda to AWS IoT Core
2+
3+
This pattern deploys an AWS Lambda function, which publishes a message to an AWS IoT Core topic. The topic is watched by a rule which will trigger an action when the condition is met. The action calls an AWS Lambda function.
4+
5+
Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns/iot-lambda-pub-receiver-cdk
6+
7+
Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
8+
9+
## Requirements
10+
11+
* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
12+
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
13+
* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
14+
* [AWS Cloud Development Kit (AWS CDK) installed](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html)
15+
16+
## Architecture diagram
17+
![Architecture diagram](./doc/architecture-diagram.png)
18+
19+
This CDK stack creates an AWS IoT Core setup with a publisher-receiver pattern using Lambda functions.
20+
Here's a breakdown of the main components:
21+
22+
1. An IoT MQTT topic named "my/mqtt/topic"
23+
24+
1. Publisher Lambda Function, written in Python and using ARM64 architecture, has permissions to publish messages to the specified MQTT topic (1). Its environment variables include the MQTT topic region and name.
25+
26+
1. Receiver Lambda Function, also in Python and using ARM64 architecture, has permissions to receive messages from the MQTT topic.
27+
28+
1. IoT Topic Rule, named "ProcessIoTMessages, uses SQL version 2016-03-23 to select all messages from the MQTT topic. It triggers the receiver Lambda function when messages arrive and it includes error logging to CloudWatch Logs.
29+
30+
## Deployment Instructions
31+
32+
1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
33+
```
34+
git clone https://github.com/aws-samples/serverless-patterns
35+
```
36+
1. Change directory to the pattern directory:
37+
```
38+
cd iot-lambda-pub-receiver-cdk
39+
```
40+
1. Install dependencies
41+
```bash
42+
npm install
43+
```
44+
1. From the command line, use AWS SAM to deploy the AWS resources for the pattern as specified in the template.yml file:
45+
```
46+
cdk deploy
47+
```
48+
49+
2. Note the outputs from the CDK deployment process. These contain the IoT endpoint address which is not relevant if you have only one account. However, in multi-accounts deployment, especially when the IoT resources are not in the same as the lambdas, then the endpoint address has to be specified in the functions' code.
50+
51+
## Testing
52+
53+
The following steps will help you test the pattern from the AWS Console:
54+
1. Trigger the publisher Lambda function
55+
1. Navigate to AWS Lambda service and open the `LambdaIotCdkStack-iotPubHandler*` function.
56+
1. From the `Test` tab, generate any event
57+
1. A green notification should appears detailing that the execution has succeeded
58+
1. Navigate to Amazon CloudWatch and open the Log group `/aws/lambda/pub-lambda`. The latest events will display the MQTT topic name and the AWS region where the message is sent.
59+
1. Verify receiver execution
60+
1. Navigate to AWS Lambda service and open the `LambdaIotCdkStack-iotReceiverHandler*` function.
61+
1. From the `Monitor` tab, look for the Invocations metrics. A data point should be displayed indicated the execution of the receiver Lambda function just after that the publisher Lambda function has been triggered.
62+
1. Navigate to Amazon CloudWatch and open the Log group `/aws/lambda/receiver-lambda`. The latest events will display the input of the publisher Lambda function.
63+
64+
## Cleanup
65+
66+
Run the given command to delete the resources that were created. It might take some time for the CloudFormation stack to get deleted.
67+
68+
```bash
69+
cdk destroy
70+
```
71+
----
72+
Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved.
73+
74+
SPDX-License-Identifier: MIT-0
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/usr/bin/env node
2+
import 'source-map-support/register';
3+
import * as cdk from 'aws-cdk-lib';
4+
import { LambdaIotCdkStack } from '../lib/lambda-iot-cdk-stack';
5+
6+
const app = new cdk.App();
7+
new LambdaIotCdkStack(app, 'LambdaIotCdkStack', { });
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
{
2+
"app": "npx ts-node --prefer-ts-exts bin/lambda-iot-cdk.ts",
3+
"watch": {
4+
"include": [
5+
"**"
6+
],
7+
"exclude": [
8+
"README.md",
9+
"cdk*.json",
10+
"**/*.d.ts",
11+
"**/*.js",
12+
"tsconfig.json",
13+
"package*.json",
14+
"yarn.lock",
15+
"node_modules",
16+
"test"
17+
]
18+
},
19+
"context": {
20+
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
21+
"@aws-cdk/core:checkSecretUsage": true,
22+
"@aws-cdk/core:target-partitions": [
23+
"aws",
24+
"aws-cn"
25+
],
26+
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
27+
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
28+
"@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
29+
"@aws-cdk/aws-iam:minimizePolicies": true,
30+
"@aws-cdk/core:validateSnapshotRemovalPolicy": true,
31+
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
32+
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
33+
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
34+
"@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
35+
"@aws-cdk/core:enablePartitionLiterals": true,
36+
"@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
37+
"@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
38+
"@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
39+
"@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
40+
"@aws-cdk/aws-route53-patters:useCertificate": true,
41+
"@aws-cdk/customresources:installLatestAwsSdkDefault": false,
42+
"@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
43+
"@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
44+
"@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
45+
"@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
46+
"@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
47+
"@aws-cdk/aws-redshift:columnId": true,
48+
"@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
49+
"@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
50+
"@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
51+
"@aws-cdk/aws-kms:aliasNameRef": true,
52+
"@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
53+
"@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
54+
"@aws-cdk/aws-efs:denyAnonymousAccess": true,
55+
"@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
56+
"@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
57+
"@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
58+
"@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
59+
"@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true,
60+
"@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true,
61+
"@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true,
62+
"@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true,
63+
"@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true,
64+
"@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true,
65+
"@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true,
66+
"@aws-cdk/aws-eks:nodegroupNameAttribute": true,
67+
"@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true,
68+
"@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true,
69+
"@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false,
70+
"@aws-cdk/aws-s3:keepNotificationInImportedBucket": false,
71+
"@aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions": true,
72+
"@aws-cdk/aws-dynamodb:resourcePolicyPerReplica": true,
73+
"@aws-cdk/aws-ec2:ec2SumTImeoutEnabled": true,
74+
"@aws-cdk/aws-appsync:appSyncGraphQLAPIScopeLambdaPermission": true,
75+
"@aws-cdk/aws-rds:setCorrectValueForDatabaseInstanceReadReplicaInstanceResourceId": true,
76+
"@aws-cdk/core:cfnIncludeRejectComplexResourceUpdateCreatePolicyIntrinsics": true,
77+
"@aws-cdk/aws-lambda-nodejs:sdkV3ExcludeSmithyPackages": true,
78+
"@aws-cdk/aws-stepfunctions-tasks:fixRunEcsTaskPolicy": true
79+
}
80+
}
26.1 KB
Loading
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
{
2+
"title": "Lambda to IoT Core to Lambda",
3+
"description": "Create a Lambda publishing into IoT topic, triggering an action calling another Lambda.",
4+
"language": "Python",
5+
"level": "200",
6+
"framework": "CDK",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This sample project demonstrates how to use an AWS Lambda Function to publish messages into AWS IoT Core topic and get a rule triggered by a condition to process relevant messages into another Lambda. This pattern is leveraging CloudWatch logs to ease the debugging and visibility into the processed messages.",
11+
"The function are written in Python and the infrastructure is described with AWS CDKv2 in Typescript. The patterns also shows an effective way to manage Python dependencies through Docker built image and requiements.txt file.",
12+
"This pattern deploys 2 Lambda functions, 1 IoT rule and action, 3 log groups."
13+
]
14+
},
15+
"gitHub": {
16+
"template": {
17+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/sfn-athena-cdk-python",
18+
"templateURL": "serverless-patterns/sfn-athena-cdk-python",
19+
"projectFolder": "iot-lambda-pub-receiver-cdk",
20+
"templateFile": "lambda-iot-cdk-stack.ts"
21+
}
22+
},
23+
"resources": {
24+
"bullets": [
25+
{
26+
"text": "AWS IoT Core - Getting started with AWS IoT Core tutorials",
27+
"link": "https://docs.aws.amazon.com/iot/latest/developerguide/iot-gs.html"
28+
},
29+
{
30+
"text": "AWS IoT Core action resources",
31+
"link": "https://docs.aws.amazon.com/iot/latest/developerguide/iot-action-resources.html"
32+
},
33+
{
34+
"test": "Building Lambda with Python",
35+
"link": "https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html"
36+
},
37+
{
38+
"text": "Working with the AWS CDK in TypeScript",
39+
"link": "https://docs.aws.amazon.com/cdk/v2/guide/work-with-cdk-typescript.html"
40+
}
41+
]
42+
},
43+
"deploy": {
44+
"text": [
45+
"cdk deploy"
46+
]
47+
},
48+
"testing": {
49+
"text": [
50+
"See the GitHub repo for detailed testing instructions."
51+
]
52+
},
53+
"cleanup": {
54+
"text": [
55+
"Delete the stack: <code>cdk destroy</code>."
56+
]
57+
},
58+
"authors": [
59+
{
60+
"name": "Philippe Wanner",
61+
"image": "https://serverlessland.com/assets/images/resources/contributors/philippe-wanner.jpg",
62+
"bio": "Philippe is a Senior Specialist Solutions Architect at Amazon Web Services based in Zurich, Switzerland. His role is to spread the migration and modernization best practices for large organisations.",
63+
"linkedin": "philippe-wanner"
64+
}
65+
]
66+
}
Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
import { CloudWatchLogsAction, LambdaFunctionAction } from '@aws-cdk/aws-iot-actions-alpha';
2+
import { IotSql, TopicRule } from '@aws-cdk/aws-iot-alpha';
3+
import { CfnOutput, RemovalPolicy, Stack, StackProps } from 'aws-cdk-lib';
4+
import { Effect, PolicyStatement, ServicePrincipal } from 'aws-cdk-lib/aws-iam';
5+
import { Architecture, Code, Function, Runtime } from 'aws-cdk-lib/aws-lambda';
6+
import { LogGroup, RetentionDays } from 'aws-cdk-lib/aws-logs';
7+
import { AwsCustomResource, AwsCustomResourcePolicy, PhysicalResourceId } from 'aws-cdk-lib/custom-resources';
8+
import { Construct } from 'constructs';
9+
10+
export class LambdaIotCdkStack extends Stack {
11+
constructor(scope: Construct, id: string, props?: StackProps) {
12+
super(scope, id, props);
13+
14+
let mqttTopicName = "my/mqtt/topic"
15+
let mqttTopicRegion = Stack.of(this).region
16+
let mqttTopicAccount = Stack.of(this).account
17+
let iotEndpointAddress = this.getIoTEndpoint().getResponseField('endpointAddress')
18+
19+
// Publisher lambda function.
20+
// Remark - if the Lambda is in the same account and region as the IoT Core endpoint, then setting the endpoint is optional.
21+
const iotPubPermission = new PolicyStatement(({
22+
effect: Effect.ALLOW,
23+
resources: [ `arn:aws:iot:${mqttTopicRegion}:${mqttTopicAccount}:topic/${mqttTopicName}` ],
24+
actions: [ "iot:Publish" ]
25+
}));
26+
const iotPubLambda = new Function(this, 'iotPubHandler', {
27+
handler: 'publisher_function.handler',
28+
code: Code.fromAsset('./src'),
29+
description: 'This function publishes a message to AWS IoT Core - MTTQ',
30+
runtime: Runtime.PYTHON_3_12,
31+
architecture: Architecture.ARM_64,
32+
logGroup: this.addLogGroup(`/aws/lambda/pub-lambda`),
33+
environment: {
34+
MQTT_TOPIC_REGION: mqttTopicRegion,
35+
MQTT_TOPIC_NAME: mqttTopicName
36+
}
37+
})
38+
iotPubLambda.addToRolePolicy(iotPubPermission)
39+
40+
// Receiver lambda function
41+
const iotReceiverPermission = new PolicyStatement(({
42+
effect: Effect.ALLOW,
43+
resources: [ `arn:aws:iot:${mqttTopicRegion}:${mqttTopicAccount}:topic/${mqttTopicName}` ],
44+
actions: [
45+
"iot:Receive"
46+
]
47+
}));
48+
const iotReceiverLambda = new Function(this, 'iotReceiverHandler', {
49+
handler: 'receiver_function.handler',
50+
description: 'This function get invoked by AWS IoT Core through the action-rule',
51+
code: Code.fromAsset('./src', {
52+
bundling: {
53+
image: Runtime.PYTHON_3_12.bundlingImage,
54+
command: [
55+
'bash', '-c',
56+
'pip install -r receiver_requirements.txt -t /asset-output && cp -au . /asset-output'
57+
],
58+
},
59+
}),
60+
runtime: Runtime.PYTHON_3_12,
61+
architecture: Architecture.ARM_64,
62+
logGroup: this.addLogGroup(`/aws/lambda/receiver-lambda`)
63+
})
64+
iotReceiverLambda.addToRolePolicy(iotReceiverPermission)
65+
66+
// Topic rule
67+
const errorLogGroup = new LogGroup(this, 'RuleErrorLogGroup', {
68+
logGroupName: '/aws/iot/rule-error-logs',
69+
retention: RetentionDays.FIVE_DAYS,
70+
removalPolicy: RemovalPolicy.DESTROY
71+
})
72+
let topicRule = new TopicRule(this, 'IoTTopicRule', {
73+
topicRuleName: 'ProcessIoTMessages',
74+
description: 'Invokes the lambda function',
75+
sql: IotSql.fromStringAsVer20160323("SELECT * FROM 'my/mqtt/topic'"),
76+
actions: [ new LambdaFunctionAction(iotReceiverLambda) ],
77+
errorAction: new CloudWatchLogsAction(errorLogGroup)
78+
})
79+
80+
// Grant permission for AWS IoT to invoke the Lambda function
81+
const iotServicePrincipal = new ServicePrincipal('iot.amazonaws.com');
82+
iotReceiverLambda.grantInvoke(iotServicePrincipal);
83+
84+
// Outputs
85+
new CfnOutput(this, "IoT Endpoint Address", {
86+
value: iotEndpointAddress ?? "Error: can't get the IoT Endpoint Address!",
87+
});
88+
}
89+
90+
// Utility function to return a log-group object
91+
private addLogGroup(logGroupName: string) {
92+
const retentionDays = RetentionDays.FIVE_DAYS
93+
const removalPolicy = RemovalPolicy.DESTROY
94+
const props = { logGroupName, retentionDays, removalPolicy }
95+
return new LogGroup(this, `${logGroupName}`, props)
96+
}
97+
98+
// Get the current account IoT-Endpoint
99+
private getIoTEndpoint() {
100+
const ioTEndpoint = new AwsCustomResource(this, 'IoTEndpoint', {
101+
onCreate: {
102+
service: 'Iot',
103+
action: 'describeEndpoint',
104+
physicalResourceId: PhysicalResourceId.fromResponse('endpointAddress'),
105+
parameters: {
106+
"endpointType": "iot:Data-ATS"
107+
}
108+
},
109+
policy: AwsCustomResourcePolicy.fromSdkCalls({resources: AwsCustomResourcePolicy.ANY_RESOURCE})
110+
})
111+
const IOT_ENDPOINT = ioTEndpoint.getResponseField('endpointAddress')
112+
return ioTEndpoint
113+
}
114+
}
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{
2+
"name": "lambda-iot-cdk",
3+
"version": "0.1.0",
4+
"bin": {
5+
"lambda-iot-cdk": "bin/lambda-iot-cdk.js"
6+
},
7+
"scripts": {
8+
"build": "tsc",
9+
"watch": "tsc -w",
10+
"cdk": "cdk"
11+
},
12+
"devDependencies": {
13+
"aws-cdk": "^2.173.0",
14+
"ts-node": "^10.9.2",
15+
"typescript": "~5.7.2"
16+
},
17+
"dependencies": {
18+
"@aws-cdk/aws-iot-actions-alpha": "^2.173.0-alpha.0",
19+
"@aws-cdk/aws-iot-alpha": "^2.173.0-alpha.0",
20+
"aws-cdk-lib": "2.173.0",
21+
"constructs": "^10.0.0",
22+
"source-map-support": "^0.5.21"
23+
}
24+
}

0 commit comments

Comments
 (0)