Skip to content

Commit c44a7f0

Browse files
JohnEllwoodBJSSJohnEllwoodBJSS
authored
Feature/tex 4957 certificate monitoring TF example (#231)
1 parent 785cfdb commit c44a7f0

File tree

8 files changed

+317
-1
lines changed

8 files changed

+317
-1
lines changed

‎practices/observability.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@
3737
* Consider using structured (i.e. Json formatted) log messages, as log aggregation systems can often perform more effective searches of these.
3838
* Tracing can be implemented using cloud platform-native tools like [AWS X-Ray](https://aws.amazon.com/xray/) or open source equivalents such as [OpenTracing](https://opentracing.io/docs/overview/what-is-tracing/). APM tools mentioned elsewhere also typically include tracing functionality.
3939
* More **things to monitor**.
40-
* Monitor (and generate alerts for) the expiry dates of the SSL certificates within the service.
40+
* Monitor (and generate alerts for) the expiry dates of the SSL certificates within the service. See [acm-cert-monitor](../tools/acm-cert-monitor/) for an example lambda and Terraform stack to monitor your AWS ACM certificates.
4141
* Subscribe to service alerts from your cloud vendors, e.g. the service-status RSS feeds for [AWS](https://status.aws.amazon.com) and [Azure](https://status.azure.com/status/).
4242
* Ensure you have reporting and alerting for the health of any services/components your service relies on, e.g. shared network connections or shared authentication services.
4343
* **Secret / sensitive data**.

‎tools/acm-cert-monitor/README.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
The Terraform 0.13.6 example here deploys a stack containing:
2+
3+
* a lambda to monitor ACM certs in the same AWS account
4+
* SNS topic and cloudwatch event rule for sending email notifications
5+
6+
The lambda also sends slack notifications but you will need to:
7+
8+
* add a secret to AWS Secrets Manager with the Slack webhook URL
9+
* update the Slack channel & secret id in the python script:
10+
11+
`"channel": "<INSERT SLACK CHANNEL>",`
12+
13+
`pw_response = boto3.client('secretsmanager', region_name=AWS_REGION).get_secret_value(SecretId='<INSERT SECRET ID>')`
14+
15+
You will also need to set the appropriate values for the variables defined in variables.tf in the .tfvars file for your environment(s)
16+
17+
As it stands, the lambda warns when certs have 45 days left and sends a critical notification at 15 days until expiry.

‎tools/acm-cert-monitor/main.tf‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
##################
2+
# Lambda
3+
##################
4+
data "archive_file" "acm_cert_monitor_zip" {
5+
type = "zip"
6+
source_dir = "${path.module}/resources"
7+
output_path = "${path.module}/.terraform/archive_files/acm_cert_monitor.zip"
8+
}
9+
10+
resource "aws_lambda_function" "acm_cert_monitor" {
11+
filename = data.archive_file.acm_cert_monitor_zip.output_path
12+
function_name = "${var.name_prefix}-acm-cert-monitor"
13+
role = aws_iam_role.acm_cert_monitor_role.arn
14+
handler = "acm_cert_monitor.lambda_handler"
15+
source_code_hash = data.archive_file.acm_cert_monitor_zip.output_base64sha256
16+
runtime = "python3.7"
17+
timeout = "180"
18+
19+
environment {
20+
variables = {
21+
SNS_TOPIC_ARN = aws_sns_topic.acm_cert_monitor_sns.arn
22+
}
23+
}
24+
}
25+
26+
##################
27+
# Event
28+
##################
29+
resource "aws_cloudwatch_event_rule" "acm_expiry_notification" {
30+
name = "${var.name_prefix}-acm-cert-expiry-monitor"
31+
description = "Event to monitor ACM expiry certs"
32+
event_pattern = <<EOF
33+
{
34+
"source": [
35+
"aws.acm"
36+
],
37+
"detail-type": [
38+
"ACM Certificate Approaching Expiration"
39+
]
40+
}
41+
EOF
42+
}
43+
44+
resource "aws_cloudwatch_event_target" "acm_cert_monitor_lambda" {
45+
target_id = "SendToLambda"
46+
rule = aws_cloudwatch_event_rule.acm_expiry_notification.name
47+
arn = aws_lambda_function.acm_cert_monitor.arn
48+
}
49+
50+
resource "aws_lambda_permission" "acm_cloudwatch_invoke_function" {
51+
statement_id = "AllowACMLambdaExecutionFromCloudWatchEvent"
52+
action = "lambda:InvokeFunction"
53+
function_name = aws_lambda_function.acm_cert_monitor.function_name
54+
principal = "events.amazonaws.com"
55+
source_arn = aws_cloudwatch_event_rule.acm_expiry_notification.arn
56+
}
57+
58+
##################
59+
# Notification
60+
##################
61+
resource "aws_sns_topic" "acm_cert_monitor_sns" {
62+
name = "${var.name_prefix}-acm-cert-monitor"
63+
display_name = var.name_prefix
64+
kms_master_key_id = "alias/aws/sns"
65+
}
66+
67+
##################
68+
# Role/Policies
69+
##################
70+
resource "aws_iam_role" "acm_cert_monitor_role" {
71+
path = "/"
72+
name = "${var.name_prefix}-acm-cert-monitor-role"
73+
74+
assume_role_policy = <<EOF
75+
{
76+
"Version": "2012-10-17",
77+
"Statement": [
78+
{
79+
"Effect": "Allow",
80+
"Principal": {
81+
"Service": [
82+
"lambda.amazonaws.com"
83+
]
84+
},
85+
"Action": [
86+
"sts:AssumeRole"
87+
]
88+
}
89+
]
90+
}
91+
EOF
92+
93+
}
94+
95+
resource "aws_iam_role_policy" "acm_cert_monitor_policy" {
96+
role = aws_iam_role.acm_cert_monitor_role.id
97+
name = "${var.name_prefix}-acm-cert-monitor-policy"
98+
99+
policy = <<EOF
100+
{
101+
"Version": "2012-10-17",
102+
"Statement": [
103+
{
104+
"Effect": "Allow",
105+
"Action": ["logs:*"],
106+
"Resource": "arn:aws:logs:*:*:*"
107+
},
108+
{
109+
"Effect": "Allow",
110+
"Action": [
111+
"acm:DescribeCertificate",
112+
"acm:GetCertificate",
113+
"acm:ListCertificates",
114+
"acm:ListTagsForCertificate"
115+
],
116+
"Resource": "*"
117+
},
118+
{
119+
"Effect": "Allow",
120+
"Action": "SNS:Publish",
121+
"Resource": "*"
122+
},
123+
{
124+
"Effect": "Allow",
125+
"Action": [
126+
"SecurityHub:BatchImportFindings",
127+
"SecurityHub:BatchUpdateFindings",
128+
"SecurityHub:DescribeHub"
129+
],
130+
"Resource": "*"
131+
},
132+
{
133+
"Effect": "Allow",
134+
"Action": "cloudwatch:ListMetrics",
135+
"Resource": "*"
136+
}
137+
]
138+
}
139+
EOF
140+
}

‎tools/acm-cert-monitor/providers.tf‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
##################################################################################
2+
# PROVIDERS
3+
##################################################################################
4+
5+
provider "aws" {
6+
profile = var.aws_profile
7+
region = var.aws_region
8+
version = "3.13.0"
9+
}
10+

‎tools/acm-cert-monitor/resources/acm_cert_monitor.py‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
# Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: MIT-0
3+
#
4+
# Permission is hereby granted, free of charge, to any person obtaining a copy of this
5+
# software and associated documentation files (the "Software"), to deal in the Software
6+
# without restriction, including without limitation the rights to use, copy, modify,
7+
# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
8+
# permit persons to whom the Software is furnished to do so.
9+
#
10+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
11+
# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
12+
# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
13+
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
14+
# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
15+
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
16+
17+
import json
18+
import urllib3
19+
import boto3
20+
import os
21+
from datetime import datetime, timedelta, timezone
22+
23+
# -------------------------------------------
24+
# setup global data
25+
# -------------------------------------------
26+
http = urllib3.PoolManager()
27+
utc = timezone.utc
28+
date = datetime.now().date()
29+
today = datetime.now().replace(tzinfo=utc)
30+
AWS_REGION = 'eu-west-2'
31+
32+
if os.environ.get('EXPIRY_DAYS') is None:
33+
expiry_days = 45
34+
else:
35+
expiry_days = int(os.environ['EXPIRY_DAYS'])
36+
37+
if os.environ.get('CRITICAL_EXPIRY_DAYS') is None:
38+
critical_expiry_days = 15
39+
else:
40+
critical_expiry_days = int(os.environ['CRITICAL_EXPIRY_DAYS'])
41+
42+
expiry_window = date + timedelta(days=expiry_days)
43+
critical_expiry_window = today + timedelta(days=critical_expiry_days)
44+
45+
46+
def lambda_handler(event, context):
47+
# if this is coming from the ACM event, its for a single certificate
48+
if event['detail-type'] == "ACM Certificate Approaching Expiration":
49+
response = handle_single_cert(event)
50+
return {
51+
'statusCode': 200,
52+
'body': response
53+
}
54+
55+
56+
def handle_single_cert(event):
57+
cert_client = boto3.client('acm')
58+
cert_details = cert_client.describe_certificate(CertificateArn=event['resources'][0])
59+
result = check_cert(cert_details)
60+
return result
61+
62+
63+
def check_cert(cert_details):
64+
result = 'The following certificate is expiring within '
65+
66+
if cert_details['Certificate']['NotAfter'].date() == expiry_window:
67+
result = result + str(expiry_days) + ' days: ' + \
68+
cert_details['Certificate']['DomainName'] + \
69+
' (' + cert_details['Certificate']['CertificateArn'] + ') '
70+
send_msg_slack(result)
71+
send_email(result)
72+
elif cert_details['Certificate']['NotAfter'] < critical_expiry_window:
73+
result = ':bomb: CRITICAL:' + result + str(critical_expiry_days) + ' days: ' + \
74+
cert_details['Certificate']['DomainName'] + \
75+
' (' + cert_details['Certificate']['CertificateArn'] + ') '
76+
send_msg_slack(result)
77+
send_email(result)
78+
79+
return result
80+
81+
82+
def send_email(result):
83+
sns_client = boto3.client('sns')
84+
sns_client.publish(TopicArn=os.environ['SNS_TOPIC_ARN'], Message=result.replace(":bomb: ", ""),
85+
Subject='Certificate Expiration Notification')
86+
87+
88+
def send_msg_slack(result):
89+
90+
url= get_slack_url()
91+
if(url is None):
92+
print ('unable to send slack message, slack url not set')
93+
else:
94+
msg = {
95+
"channel": "<INSERT SLACK CHANNEL>",
96+
"username": "ACM_Expiry",
97+
"text": result,
98+
"icon_emoji": ""
99+
}
100+
101+
encoded_msg = json.dumps(msg).encode('utf-8')
102+
resp = http.request('POST', url, body=encoded_msg)
103+
print({
104+
"message": result,
105+
"status_code": resp.status,
106+
"response": resp.data
107+
})
108+
109+
def get_slack_url():
110+
slack_url = None
111+
pw_response = boto3.client('secretsmanager', region_name=AWS_REGION).get_secret_value(SecretId='<INSERT SECRET ID>')
112+
secrets = pw_response['SecretString']
113+
secret_dict= json.loads(secrets)
114+
slack_url_key = 'slack_hook_url'
115+
if slack_url_key in secret_dict:
116+
slack_url = secret_dict[slack_url_key]
117+
print('slack hook url'+slack_url)
118+
else:
119+
print(slack_url_key+' not declared in AWS Secrets')
120+
121+
return slack_url

‎tools/acm-cert-monitor/terraform.tf‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
terraform {
2+
backend "s3" {
3+
encrypt = true
4+
}
5+
}
6+

‎tools/acm-cert-monitor/variables.tf‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
############################
2+
# AWS COMMON
3+
############################
4+
variable "aws_profile" {
5+
description = "The AWS profile"
6+
}
7+
8+
variable "aws_region" {
9+
description = "The AWS region"
10+
}
11+
12+
variable "aws_account_id" {
13+
description = "AWS account Number for Athena log location"
14+
}
15+
16+
variable "name_prefix" {
17+
description = "Prefix used for naming resources"
18+
}

‎tools/acm-cert-monitor/versions.tf‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
2+
terraform {
3+
required_version = "= 0.13.6"
4+
}

0 commit comments

Comments
 (0)