Skip to content

Commit 903fe38

Browse files
julianwoodjulianwood
authored
Merge pull request #2517 from shubsmor/shubsmor-lambda-cloudwatch-subscription-sns-notification
New serverless pattern - lambda-cloudwatch-subscription-sns-notification
2 parents e68ff44 + c312a92 commit 903fe38

File tree

5 files changed

+348
-0
lines changed

5 files changed

+348
-0
lines changed

‎cloudwatch-account-subscription-Lambda-sns-notification/README.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Account level alerting using Amazon CloudWatch, AWS Lambda and Amazon SNS
2+
3+
This pattern demonstrates an alerting system that leverages an AWS CloudWatch Logs Account-level Subscription filter to trigger a Lambda function, which in turn sends an SNS notification when a specified pattern (in this case, "Exception") is matched in log events across any log group within the account. Currently, there is no out-of-the-box offering from AWS that allows customers to be alerted on specific event patterns across their entire account.
4+
5+
By implementing this pattern, customers can proactively monitor and receive notifications for critical events or errors, such as exceptions or any specific pattern, that may occur in any of their applications or services. This centralized approach to log monitoring and alerting eliminates the need to configure individual alerting mechanisms for each log group or application, thereby streamlining the process and ensuring consistent alerting across the entire AWS account.
6+
7+
Important:
8+
* This application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
9+
10+
* Additionally, There is a risk of causing an infinite recursive loop with subscription filters that can lead to a large increase in ingestion billing if not addressed. To mitigate this risk, we recommend that you use selection criteria in your account-level subscription filters to exclude log groups that ingest log data from resources that are part of the subscription delivery workflow. Please see [Account-level subscription filters risk](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters-AccountLevel.html)
11+
12+
## Requirements
13+
14+
* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
15+
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
16+
* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
17+
* [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed
18+
19+
## Deployment Instructions
20+
21+
1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
22+
```
23+
git clone https://github.com/aws-samples/serverless-patterns
24+
```
25+
2. Change directory to the pattern directory:
26+
```
27+
cd cloudwatch-account-subscription-Lambda-sns-notification
28+
```
29+
3. From the command line, use AWS SAM to deploy the AWS resources for the pattern as specified in the template.yml file:
30+
```
31+
sam build
32+
33+
sam deploy --guided --parameter-overrides EmailAddress='[email protected]' LambdaFunctionName='NotificationLambda'
34+
```
35+
4. During the prompts:
36+
* Enter a stack name
37+
* Enter the desired AWS Region
38+
* Allow SAM CLI to create IAM roles with the required permissions.
39+
40+
Once you have run `sam deploy --guided` mode once and saved arguments to a configuration file (samconfig.toml), you can use `sam deploy` in future to use these defaults.
41+
42+
5. Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs which are used for testing.
43+
44+
## How it works
45+
46+
The key components of this pattern are:
47+
48+
1. AWS CloudWatch Logs Account-level Subscription Filter: This filter is configured to match the desired pattern ("Exception" in this case) and send the matching log events to a designated AWS Lambda function.
49+
50+
2. AWS Lambda Function: This function processes the log events received from the subscription filter and sends a notification to an SNS topic when the specified pattern is matched.
51+
52+
3. AWS Simple Notification Service (SNS) Topic: This SNS topic receives the notifications from the Lambda function and delivers them to subscribed endpoints, such as email addresses or mobile devices, enabling real-time alerting for critical events.
53+
54+
55+
This pattern deploys an account-level subscription filter that monitors all existing and new log groups for a matching log pattern of "Exception". This allows you to catch any event containing the "Exception" pattern across all applications within your AWS account. Optionally, you can produce an exception in any application's logs to test this pattern.
56+
*This pattern creates a new SNS topic with an email subscription. Please confirm the email verification by clicking on the "Confirm subscription" link sent via Amazon SNS.*
57+
58+
## Testing
59+
To test this pattern, which monitors all log groups for the "Exception" pattern and sends SNS notifications, follow these steps:
60+
61+
- Choose an application generating logs in your AWS account.
62+
- Intentionally introduce an exception or error condition that logs an "Exception" message.
63+
- Verify the log event containing "Exception" is written to CloudWatch Logs.
64+
- Monitor for an SNS email notification containing details like the log group name, log stream name, and the log event with the "Exception" pattern.
65+
66+
This pattern automatically monitors new log groups created, ensuring comprehensive coverage across your AWS account.
67+
68+
## Cleanup
69+
70+
1. Delete the stack
71+
```bash
72+
aws cloudformation delete-stack --stack-name STACK_NAME
73+
```
74+
1. Confirm the stack has been deleted
75+
```bash
76+
aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus"
77+
```
78+
----
79+
Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved.
80+
81+
SPDX-License-Identifier: MIT-0

‎cloudwatch-account-subscription-Lambda-sns-notification/cloudwatch-account-subscription-Lambda-sns-notification.json‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
{
2+
"title": "Account level alerting using Amazon CloudWatch, AWS Lambda and Amazon SNS",
3+
"description": "Creates alerting system at account level which triggers Lambda function which can parse and publish a SNS notification",
4+
"language": "Python",
5+
"level": "200",
6+
"framework": "SAM",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This sample project showcases a centralized alerting system that leverages CloudWatch Logs Subscription filters and Lambda functions to trigger SNS notifications when a specified pattern is matched across any CloudWatch log group within the account. This robust solution enables comprehensive monitoring and proactive alerting, ensuring timely identification and mitigation of critical events or errors",
11+
"The core components of this pattern include a CloudWatch Logs Account-level Subscription filter, which monitors log events across the entire account, and a notification Lambda function. This function is triggered by the Subscription filter and responsibly publishes relevant details, such as log group name, log stream name, and the matched log event, to the SNS topic, ensuring that stakeholders receive comprehensive and actionable information."
12+
]
13+
},
14+
"gitHub": {
15+
"template": {
16+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/cloudwatch-account-subscription-Lambda-sns-notification",
17+
"templateURL": "serverless-patterns/cloudwatch-account-subscription-Lambda-sns-notification",
18+
"projectFolder": "cloudwatch-account-subscription-Lambda-sns-notification",
19+
"templateFile": "template.yaml"
20+
}
21+
},
22+
"resources": {
23+
"bullets": [
24+
{
25+
"text": "Account-level subscription filters",
26+
"link": "https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters-AccountLevel.html#LambdaFunctionExample-AccountLevel"
27+
},
28+
{
29+
"text": "Lambda function",
30+
"link": "https://docs.aws.amazon.com/lambda/latest/dg/welcome.html"
31+
}
32+
]
33+
},
34+
"deploy": {
35+
"text": [
36+
"sam build",
37+
"sam deploy --guided --parameter-overrides EmailAddress='[email protected]' LambdaFunctionName='NotificationLambda'"
38+
]
39+
},
40+
"testing": {
41+
"text": [
42+
"To test this pattern, which monitors all log groups for the 'Exception' pattern and sends SNS notifications, follow these steps:",
43+
"1. Choose an application generating logs in your AWS account.",
44+
"2. Intentionally introduce an exception or error condition that logs an 'Exception' message.",
45+
"3. Verify the log event containing 'Exception' is written to CloudWatch Logs.",
46+
"4. Monitor for an SNS email notification containing details like the log group name, log stream name, and the log event with the 'Exception' pattern.",
47+
"This pattern automatically monitors new log groups created, ensuring comprehensive coverage across your AWS account."
48+
]
49+
},
50+
"cleanup": {
51+
"text": [
52+
"Delete the stack: <code>cdk delete</code>."
53+
]
54+
},
55+
"authors": [
56+
{
57+
"name": "Shubham",
58+
"image": "https://avatars.githubusercontent.com/u/150242047?s=400&u=aaa2db07529d3e1e82ec59daacb05b6abc7c3a5f&v=4",
59+
"bio": "Cloud Support Engineer - SVLS",
60+
"linkedin": "shubham-more-1b6aa185"
61+
}
62+
],
63+
"patternArch": {
64+
"icon1": {
65+
"x": 20,
66+
"y": 50,
67+
"service": "cloudwatch",
68+
"label": "Amazon CloudWatch"
69+
},
70+
"icon2": {
71+
"x": 50,
72+
"y": 50,
73+
"service": "lambda",
74+
"label": "AWS Lambda"
75+
},
76+
"line1": {
77+
"from": "icon1",
78+
"to": "icon2",
79+
"label": "trigger"
80+
},
81+
"icon3": {
82+
"x": 80,
83+
"y": 50,
84+
"service": "sns",
85+
"label": "Amazon SNS"
86+
},
87+
"line2": {
88+
"from": "icon2",
89+
"to": "icon3",
90+
"label": "publish"
91+
}
92+
}
93+
}

‎cloudwatch-account-subscription-Lambda-sns-notification/example-pattern.json‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
{
2+
"title": "Account level alerting using Amazon CloudWatch, AWS Lambda and Amazon SNS",
3+
"description": "Creates alerting system at account level which triggers Lambda function which can parse and publish a SNS notification",
4+
"language": "Python",
5+
"level": "200",
6+
"framework": "SAM",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This sample project showcases a centralized alerting system that leverages CloudWatch Logs Subscription filters and Lambda functions to trigger SNS notifications when a specified pattern is matched across any CloudWatch log group within the account. This robust solution enables comprehensive monitoring and proactive alerting, ensuring timely identification and mitigation of critical events or errors",
11+
"The core components of this pattern include a CloudWatch Logs Account-level Subscription filter, which monitors log events across the entire account, and a notification Lambda function. This function is triggered by the Subscription filter and responsibly publishes relevant details, such as log group name, log stream name, and the matched log event, to the SNS topic, ensuring that stakeholders receive comprehensive and actionable information."
12+
]
13+
},
14+
"gitHub": {
15+
"template": {
16+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/cloudwatch-account-subscription-Lambda-sns-notification",
17+
"templateURL": "serverless-patterns/cloudwatch-account-subscription-Lambda-sns-notification",
18+
"projectFolder": "cloudwatch-account-subscription-Lambda-sns-notification",
19+
"templateFile": "template.yaml"
20+
}
21+
},
22+
"resources": {
23+
"bullets": [
24+
{
25+
"text": "Account-level subscription filters",
26+
"link": "https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters-AccountLevel.html#LambdaFunctionExample-AccountLevel"
27+
},
28+
{
29+
"text": "Lambda function",
30+
"link": "https://docs.aws.amazon.com/lambda/latest/dg/welcome.html"
31+
}
32+
]
33+
},
34+
"deploy": {
35+
"text": [
36+
"sam build",
37+
"sam deploy --guided --parameter-overrides EmailAddress='[email protected]' LambdaFunctionName='NotificationLambda'"
38+
]
39+
},
40+
"testing": {
41+
"text": [
42+
"To test this pattern, which monitors all log groups for the 'Exception' pattern and sends SNS notifications, follow these steps:",
43+
"1. Choose an application generating logs in your AWS account.",
44+
"2. Intentionally introduce an exception or error condition that logs an 'Exception' message.",
45+
"3. Verify the log event containing 'Exception' is written to CloudWatch Logs.",
46+
"4. Monitor for an SNS email notification containing details like the log group name, log stream name, and the log event with the 'Exception' pattern.",
47+
"This pattern automatically monitors new log groups created, ensuring comprehensive coverage across your AWS account."
48+
]
49+
},
50+
"cleanup": {
51+
"text": [
52+
"Delete the stack: <code>cdk delete</code>."
53+
]
54+
},
55+
"authors": [
56+
{
57+
"name": "Shubham",
58+
"image": "https://avatars.githubusercontent.com/u/150242047?s=400&u=aaa2db07529d3e1e82ec59daacb05b6abc7c3a5f&v=4",
59+
"bio": "Cloud Support Engineer - SVLS",
60+
"linkedin": "shubham-more-1b6aa185"
61+
}
62+
]
63+
}

‎cloudwatch-account-subscription-Lambda-sns-notification/notification-lambda/index.py‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
import json
2+
import boto3
3+
import gzip
4+
import base64
5+
from io import BytesIO
6+
import os
7+
sns = boto3.client('sns')
8+
9+
def lambda_handler(event, context):
10+
topicArn = os.environ['SNS_TOPIC_ARN']
11+
12+
compressed_log_data = event['awslogs']['data']
13+
uncompressed_log_data = gzip.decompress(base64.b64decode(compressed_log_data))
14+
print("Printing uncompressed_log_data :", uncompressed_log_data)
15+
16+
log_events = json.loads(uncompressed_log_data)['logEvents']
17+
log_group = json.loads(uncompressed_log_data)['logGroup']
18+
log_stream = json.loads(uncompressed_log_data)['logStream']
19+
20+
notification_message = f"*** An Exception has occurred within the specified log group. Please review the logs for detailed information and troubleshoot accordingly.***. \n Logdetails : \n Log events received from log group: {log_group}, log stream: {log_stream} \n log event: {log_events}"
21+
22+
response = sns.publish(
23+
TopicArn=topicArn,
24+
Message=notification_message,
25+
Subject='Account Notification'
26+
)
27+
28+
print(f"Notification published - messageID:", response['MessageId'])
29+
30+
return {
31+
"statusCode": 200,
32+
"body": json.dumps({
33+
"message": "hello world"
34+
}),
35+
}

‎cloudwatch-account-subscription-Lambda-sns-notification/template.yaml‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
AWSTemplateFormatVersion: '2010-09-09'
2+
Transform: AWS::Serverless-2016-10-31
3+
Description: SAM template to demonstrate alerting with CloudWatch Subscription Filter
4+
5+
Parameters:
6+
EmailAddress:
7+
Type: String
8+
Description: Email address to receive notifications
9+
10+
LambdaFunctionName:
11+
Type: String
12+
Description: Name of the Lambda function to be invoked by CloudWatch Logs
13+
14+
Globals:
15+
Function:
16+
Timeout: 3
17+
MemorySize: 128
18+
19+
Resources:
20+
# SNS Topic for notifications
21+
NotificationTopic:
22+
Type: AWS::SNS::Topic
23+
Properties:
24+
TopicName: AccountLevelNotificationTopic
25+
# SNS Topic Subscription for Email Notifications
26+
27+
EmailSubscription:
28+
Type: AWS::SNS::Subscription
29+
Properties:
30+
Endpoint: !Ref EmailAddress
31+
Protocol: email
32+
TopicArn: !Ref NotificationTopic
33+
34+
CloudWatchLogSubscriptionLambdaPermission:
35+
Type: AWS::Lambda::Permission
36+
Properties:
37+
FunctionName: !GetAtt NotificationLambda.Arn
38+
Principal: logs.amazonaws.com
39+
Action: 'lambda:InvokeFunction'
40+
SourceArn: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:*'
41+
SourceAccount: !Ref AWS::AccountId
42+
43+
44+
CloudWatchLogSubscriptionPolicy:
45+
Type: AWS::Logs::AccountPolicy
46+
Properties:
47+
PolicyName: CloudWatchLogsAccountLevelSubscriptionPolicy
48+
PolicyDocument: !Sub "{\"DestinationArn\":\"arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:${LambdaFunctionName}\", \"FilterPattern\": \"Exception\", \"Distribution\": \"Random\"}"
49+
SelectionCriteria: !Sub "LogGroupName NOT IN [\"/aws/lambda/example\", \"/custom/lambda/print\"]" #All the loggroups [Make sure to give valid loggroups] you would like to exclude this helps with log recursion prevention - https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Subscriptions-recursion-prevention.html
50+
PolicyType: "SUBSCRIPTION_FILTER_POLICY"
51+
Scope: "ALL"
52+
DependsOn: CloudWatchLogSubscriptionLambdaPermission
53+
54+
# Notification Lambda function
55+
NotificationLambda:
56+
Type: AWS::Serverless::Function
57+
Properties:
58+
FunctionName: !Ref LambdaFunctionName
59+
Handler: index.lambda_handler
60+
Runtime: python3.13
61+
CodeUri: notification-lambda/
62+
Environment:
63+
Variables:
64+
SNS_TOPIC_ARN: !Ref NotificationTopic
65+
Policies:
66+
- SNSPublishMessagePolicy:
67+
TopicName: !GetAtt NotificationTopic.TopicName
68+
69+
70+
Outputs:
71+
NotificationLambdaArn:
72+
Description: ARN of the Notification Lambda function
73+
Value: !GetAtt NotificationLambda.Arn
74+
NotificationTopicArn:
75+
Description: ARN of the SNS Topic for notifications
76+
Value: !Ref NotificationTopic

0 commit comments

Comments
 (0)