Merge pull request #6 from SumoLogic/aws

Modules for AWSS

Author: arunpatyal

aws/cloudtrail/README.md

@@ -13,27 +13,29 @@ This module is used to create AWS and Sumo Logic resource to collect CloudTrail
 | Name | Version |
 |------|---------|
 | terraform | >= 0.13.0 |
-| aws | ~> 3.29.1 |
-| sumologic | ~> 2.6.0 |
-| time | 0.7.1 |
+| aws | >= 3.42.0 |
+| random | >=3.1.0 |
+| sumologic | >= 2.9.0 |
+| time | >=0.7.1 |
 
 ## Providers
 
 | Name | Version |
 |------|---------|
-| aws | ~> 3.29.1 |
-| sumologic | ~> 2.6.0 |
-| time | 0.7.1 |
+| aws | >= 3.42.0 |
+| random | >=3.1.0 |
+| sumologic | >= 2.9.0 |
+| time | >=0.7.1 |
 
 ## Inputs
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| cloudtrail\_details | Provide details for the AWS CloudTrail. If not provided, then defaults will be used. | <pre>object({<br>    name                          = string<br>    is_multi_region_trail         = bool<br>    is_organization_trail         = bool<br>    include_global_service_events = bool<br>  })</pre> | <pre>{<br>  "include_global_service_events": false,<br>  "is_multi_region_trail": false,<br>  "is_organization_trail": false,<br>  "name": "SumoLogic-Terraform-CloudTrail"<br>}</pre> | no |
-| collector\_details | Provide details for the Sumo Logic collector. If not provided, then defaults will be used. | <pre>object({<br>    collector_name = string<br>    description    = string<br>    fields         = map(string)<br>  })</pre> | <pre>{<br>  "collector_name": "SumoLogic CloudTrail Collector <AWS Account Id>",<br>  "description": "This collector is created using Sumo Logic terraform AWS cloudtrail module to collect AWS cloudtrail logs.",<br>  "fields": {}<br>}</pre> | no |
+| cloudtrail\_details | Provide details for the AWS CloudTrail. If not provided, then defaults will be used. | <pre>object({<br>    name                          = string<br>    is_multi_region_trail         = bool<br>    is_organization_trail         = bool<br>    include_global_service_events = bool<br>  })</pre> | <pre>{<br>  "include_global_service_events": false,<br>  "is_multi_region_trail": false,<br>  "is_organization_trail": false,<br>  "name": "SumoLogic-Terraform-CloudTrail-random-id"<br>}</pre> | no |
+| collector\_details | Provide details for the Sumo Logic collector. If not provided, then defaults will be used. | <pre>object({<br>    collector_name = string<br>    description    = string<br>    fields         = map(string)<br>  })</pre> | <pre>{<br>  "collector_name": "SumoLogic CloudTrail Collector <Random ID>",<br>  "description": "This collector is created using Sumo Logic terraform AWS cloudtrail module to collect AWS cloudtrail logs.",<br>  "fields": {}<br>}</pre> | no |
 | create\_collector | Provide "true" if you would like to create the Sumo Logic Collector. | `bool` | n/a | yes |
 | create\_trail | Provide "true" if you would like to create the AWS CloudTrail. If the bucket is created by the module, module by default creates the AWS cloudtrail. | `bool` | n/a | yes |
-| source\_details | Provide details for the Sumo Logic CloudTrail source. If not provided, then defaults will be used. | <pre>object({<br>    source_name     = string<br>    source_category = string<br>    collector_id    = string<br>    description     = string<br>    bucket_details = object({<br>      create_bucket        = bool<br>      bucket_name          = string<br>      path_expression      = string<br>      force_destroy_bucket = bool<br>    })<br>    paused               = bool<br>    scan_interval        = string<br>    sumo_account_id      = number<br>    cutoff_relative_time = string<br>    fields               = map(string)<br>    iam_role_arn         = string<br>    sns_topic_arn        = string<br>  })</pre> | <pre>{<br>  "bucket_details": {<br>    "bucket_name": "cloudtrail-logs-accountid-region",<br>    "create_bucket": true,<br>    "force_destroy_bucket": true,<br>    "path_expression": "AWSLogs/<ACCOUNT-ID>/CloudTrail/<REGION-NAME>/*"<br>  },<br>  "collector_id": "",<br>  "cutoff_relative_time": "-1d",<br>  "description": "This source is created using Sumo Logic terraform AWS cloudtrail module to collect AWS cloudtrail logs.",<br>  "fields": {},<br>  "iam_role_arn": "",<br>  "paused": false,<br>  "scan_interval": 300000,<br>  "sns_topic_arn": "",<br>  "source_category": "Labs/aws/cloudtrail",<br>  "source_name": "CloudTrail Source",<br>  "sumo_account_id": 926226587429<br>}</pre> | no |
+| source\_details | Provide details for the Sumo Logic CloudTrail source. If not provided, then defaults will be used. | <pre>object({<br>    source_name     = string<br>    source_category = string<br>    collector_id    = string<br>    description     = string<br>    bucket_details = object({<br>      create_bucket        = bool<br>      bucket_name          = string<br>      path_expression      = string<br>      force_destroy_bucket = bool<br>    })<br>    paused               = bool<br>    scan_interval        = string<br>    sumo_account_id      = number<br>    cutoff_relative_time = string<br>    fields               = map(string)<br>    iam_role_arn         = string<br>    sns_topic_arn        = string<br>  })</pre> | <pre>{<br>  "bucket_details": {<br>    "bucket_name": "cloudtrail-logs-random-id",<br>    "create_bucket": true,<br>    "force_destroy_bucket": true,<br>    "path_expression": "AWSLogs/<ACCOUNT-ID>/CloudTrail/<REGION-NAME>/*"<br>  },<br>  "collector_id": "",<br>  "cutoff_relative_time": "-1d",<br>  "description": "This source is created using Sumo Logic terraform AWS cloudtrail module to collect AWS cloudtrail logs.",<br>  "fields": {},<br>  "iam_role_arn": "",<br>  "paused": false,<br>  "scan_interval": 300000,<br>  "sns_topic_arn": "",<br>  "source_category": "Labs/aws/cloudtrail",<br>  "source_name": "CloudTrail Source",<br>  "sumo_account_id": 926226587429<br>}</pre> | no |
 | sumologic\_organization\_id | Appears on the Account Overview page that displays information about your Sumo Logic organization. Used for IAM Role in Sumo Logic AWS Sources. | `string` | n/a | yes |
 
 ## Outputs
@@ -46,5 +48,6 @@ This module is used to create AWS and Sumo Logic resource to collect CloudTrail
 | aws\_s3\_bucket\_notification | AWS S3 Bucket Notification attached to the AWS S3 Bucket |
 | aws\_sns\_subscription | AWS SNS subscription to Sumo Logic AWS CloudTrail source. |
 | aws\_sns\_topic | AWS SNS topic attached to the AWS S3 bucket. |
+| random\_string | Random String value created. |
 | sumologic\_collector | Sumo Logic hosted collector. |
 | sumologic\_source | Sumo Logic AWS CloudTrail source. |

aws/cloudtrail/cloudtrail.tf

@@ -6,6 +6,12 @@
 # 5. Create the source either in the collector created or in the collector id provided.
 # 6. Create SNS Subscription to be attached to the source and SNS Topic.
 
+resource "random_string" "aws_random" {
+  length  = 10
+  special = false
+  upper   = false
+}
+
 resource "aws_s3_bucket" "s3_bucket" {
   for_each = toset(var.source_details.bucket_details.create_bucket ? ["s3_bucket"] : [])
 
@@ -20,11 +26,11 @@ resource "aws_s3_bucket" "s3_bucket" {
 resource "aws_sns_topic" "sns_topic" {
   for_each = toset(local.create_sns_topic ? ["sns_topic"] : [])
 
-  name = "SumoLogic-Terraform-CloudTrail-Module-${local.aws_account_id}"
+  name = "SumoLogic-Terraform-CloudTrail-Module-${random_string.aws_random.id}"
   policy = templatefile("${path.module}/templates/sns_topic_policy.tmpl", {
     BUCKET_NAME    = local.bucket_name,
     AWS_REGION     = local.aws_region,
-    SNS_TOPIC_NAME = "SumoLogic-Terraform-CloudTrail-Module-${local.aws_account_id}",
+    SNS_TOPIC_NAME = "SumoLogic-Terraform-CloudTrail-Module-${random_string.aws_random.id}",
     AWS_ACCOUNT    = local.aws_account_id
   })
 }
@@ -43,7 +49,7 @@ resource "aws_s3_bucket_notification" "bucket_notification" {
 resource "aws_cloudtrail" "cloudtrail" {
   for_each = toset(local.create_trail ? ["cloudtrail"] : [])
 
-  name                          = var.cloudtrail_details.name
+  name                          = local.cloudtrail_name
   include_global_service_events = var.cloudtrail_details.include_global_service_events
   s3_bucket_name                = var.source_details.bucket_details.create_bucket ? aws_s3_bucket.s3_bucket["s3_bucket"].id : local.bucket_name
   is_multi_region_trail         = var.cloudtrail_details.is_multi_region_trail
@@ -53,7 +59,7 @@ resource "aws_cloudtrail" "cloudtrail" {
 resource "aws_iam_role" "source_iam_role" {
   for_each = toset(local.create_iam_role ? ["source_iam_role"] : [])
 
-  name = "SumoLogic-Terraform-CloudTrail-Module-${local.aws_account_id}-${local.aws_region}"
+  name = "SumoLogic-Terraform-CloudTrail-Module-${random_string.aws_random.id}"
   path = "/"
 
   assume_role_policy = templatefile("${path.module}/templates/sumologic_assume_role.tmpl", {
@@ -68,7 +74,7 @@ resource "aws_iam_role" "source_iam_role" {
 resource "aws_iam_policy" "iam_policy" {
   for_each = toset(local.create_iam_role ? ["iam_policy"] : [])
 
-  name = "SumoLogicCloudTrailSource-${local.aws_account_id}-${local.aws_region}"
+  name = "SumoLogicCloudTrailSource-${random_string.aws_random.id}"
   policy = templatefile("${path.module}/templates/sumologic_source_policy.tmpl", {
     BUCKET_NAME = local.bucket_name
   })

aws/cloudtrail/locals.tf

@@ -5,10 +5,13 @@ locals {
   aws_region = data.aws_region.current.id
 
   # Get the default collector name if no collector name is provided.
-  collector_name = var.collector_details.collector_name == "SumoLogic CloudTrail Collector <AWS Account Id>" ? "SumoLogic CloudTrail Collector ${local.aws_account_id}" : var.collector_details.collector_name
+  collector_name = var.collector_details.collector_name == "SumoLogic CloudTrail Collector <Random ID>" ? "SumoLogic CloudTrail Collector ${random_string.aws_random.id}" : var.collector_details.collector_name
+
+  # Get the default cloudtrail name if default is provided.
+  cloudtrail_name = var.cloudtrail_details.name == "SumoLogic-Terraform-CloudTrail-random-id" ? "SumoLogic-Terraform-CloudTrail-${random_string.aws_random.id}" : var.cloudtrail_details.name
 
   # Get the default bucket name when no bucket is provided and create_bucket is true.
-  bucket_name = var.source_details.bucket_details.create_bucket && var.source_details.bucket_details.bucket_name == "cloudtrail-logs-accountid-region" ? "cloudtrail-logs-${local.aws_account_id}-${local.aws_region}" : var.source_details.bucket_details.bucket_name
+  bucket_name = var.source_details.bucket_details.create_bucket && var.source_details.bucket_details.bucket_name == "cloudtrail-logs-random-id" ? "cloudtrail-logs-${random_string.aws_random.id}" : var.source_details.bucket_details.bucket_name
 
   # Create IAM role condition if no IAM ROLE ARN is provided.
   create_iam_role = var.source_details.iam_role_arn != "" ? false : true

aws/cloudtrail/outputs.tf

@@ -1,3 +1,8 @@
+output "random_string" {
+  value       = random_string.aws_random
+  description = "Random String value created."
+}
+
 output "aws_s3_bucket" {
   value       = var.source_details.bucket_details.create_bucket ? aws_s3_bucket.s3_bucket : {}
   description = "AWS S3 Bucket name created to Store the CloudTrail logs."

aws/cloudtrail/variables.tf

@@ -11,7 +11,7 @@ variable "collector_details" {
   })
   description = "Provide details for the Sumo Logic collector. If not provided, then defaults will be used."
   default = {
-    collector_name = "SumoLogic CloudTrail Collector <AWS Account Id>"
+    collector_name = "SumoLogic CloudTrail Collector <Random ID>"
     description    = "This collector is created using Sumo Logic terraform AWS cloudtrail module to collect AWS cloudtrail logs."
     fields         = {}
   }
@@ -45,7 +45,7 @@ variable "source_details" {
     collector_id    = ""
     bucket_details = {
       create_bucket        = true
-      bucket_name          = "cloudtrail-logs-accountid-region"
+      bucket_name          = "cloudtrail-logs-random-id"
       path_expression      = "AWSLogs/<ACCOUNT-ID>/CloudTrail/<REGION-NAME>/*"
       force_destroy_bucket = true
     }
@@ -91,7 +91,7 @@ variable "cloudtrail_details" {
   })
   description = "Provide details for the AWS CloudTrail. If not provided, then defaults will be used."
   default = {
-    name                          = "SumoLogic-Terraform-CloudTrail"
+    name                          = "SumoLogic-Terraform-CloudTrail-random-id"
     is_multi_region_trail         = false
     is_organization_trail         = false
     include_global_service_events = false

aws/cloudtrail/versions.tf

@@ -4,15 +4,19 @@ terraform {
   required_providers {
     aws = {
       source  = "hashicorp/aws"
-      version = "~> 3.29.1"
+      version = ">= 3.42.0"
     }
     sumologic = {
-      version = "~> 2.6.0"
+      version = ">= 2.9.0"
       source  = "SumoLogic/sumologic"
     }
     time = {
       source  = "hashicorp/time"
-      version = "0.7.1"
+      version = ">=0.7.1"
+    }
+    random = {
+      source  = "hashicorp/random"
+      version = ">=3.1.0"
     }
   }
 }

aws/cloudwatchlogsforwarder/README.md

@@ -0,0 +1,54 @@
+# SumoLogic-AWS-CloudWatchLogsForwarder
+
+This module is used to create the SumoLogic AWS HTTP source to collect AWS CloudWatch logs. Features include
+- Create AWS resources to setup IAM Roles, SQS, SNS, Metric Alarm, Lambda functions.
+- Create Sumo Logic hosted collector. Existing collector can also be used.
+- Create Sumo Logic HTTP Source for logs.
+- Auto enable logs subscription for Existing and New log groups after installing the module.
+
+## Requirements
+
+| Name | Version |
+|------|---------|
+| terraform | >= 0.13.0 |
+| aws | >= 3.42.0 |
+| random | >= 3.1.0 |
+| sumologic | >= 2.9.0 |
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| aws | >= 3.42.0 |
+| random | >= 3.1.0 |
+| sumologic | >= 2.9.0 |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| auto\_enable\_logs\_subscription | New - Automatically subscribes new log groups to send logs to Sumo Logic.<br>                              Existing - Automatically subscribes existing log groups to send logs to Sumo Logic.<br>                           Both - Automatically subscribes new and existing log groups.<br>                                None - Skips Automatic subscription. | `string` | `"Both"` | no |
+| auto\_enable\_logs\_subscription\_options | filter - Enter regex for matching logGroups. Regex will check for the name. Visit https://help.sumologic.com/03Send-Data/Collect-from-Other-Data-Sources/Auto-Subscribe_AWS_Log_Groups_to_a_Lambda_Function#Configuring_parameters | <pre>object({<br>    filter = string<br>  })</pre> | <pre>{<br>  "filter": "lambda"<br>}</pre> | no |
+| collector\_details | Provide details for the Sumo Logic collector. If not provided, then defaults will be used. | <pre>object({<br>    collector_name = string<br>    description    = string<br>    fields         = map(string)<br>  })</pre> | <pre>{<br>  "collector_name": "SumoLogic CloudWatch Logs Collector <Random ID>",<br>  "description": "This collector is created using Sumo Logic terraform AWS CloudWatch Logs forwarder to collect AWS cloudwatch logs.",<br>  "fields": {}<br>}</pre> | no |
+| create\_collector | Provide "true" if you would like to create the Sumo Logic Collector. | `bool` | n/a | yes |
+| email\_id | Email for receiving alerts. A confirmation email is sent after the deployment is complete. It can be confirmed to subscribe for alerts. | `string` | `"[email protected]"` | no |
+| include\_log\_group\_info | Enable loggroup/logstream values in logs. | `bool` | `true` | no |
+| log\_format | Service for Cloudwatch logs source. | `string` | `"Others"` | no |
+| log\_stream\_prefix | LogStream name prefixes to filter by logStream. Please note this is separate from a logGroup. This is used only to send certain logStreams within a Cloudwatch logGroup(s). LogGroups still need to be subscribed to the created Lambda function regardless of this input value. | `list(string)` | `[]` | no |
+| source\_details | Provide details for the Sumo Logic HTTP source. If not provided, then defaults will be used. | <pre>object({<br>    source_name     = string<br>    source_category = string<br>    collector_id    = string<br>    description     = string<br>    fields          = map(string)<br>  })</pre> | <pre>{<br>  "collector_id": "",<br>  "description": "This source is created using Sumo Logic terraform AWS CloudWatch Logs forwarder to collect AWS cloudwatch logs.",<br>  "fields": {},<br>  "source_category": "Labs/aws/cloudwatch",<br>  "source_name": "CloudWatch Logs Source"<br>}</pre> | no |
+| workers | Number of lambda function invocations for Cloudwatch logs source Dead Letter Queue processing. | `number` | `4` | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| aws\_cloudwatch\_log\_group | AWS Log group created to attach to the lambda function. |
+| aws\_cloudwatch\_metric\_alarm | AWS CLoudWatch metric alarm. |
+| aws\_cw\_lambda\_function | AWS Lambda fucntion to send logs to Sumo Logic. |
+| aws\_iam\_role | AWS IAM role with permission to setup lambda. |
+| aws\_serverlessapplicationrepository\_cloudformation\_stack | AWS CloudFormation stack for Auto Enable logs subscription. |
+| aws\_sns\_topic | AWS SNS topic |
+| aws\_sqs\_queue | AWS SQS queue to Store the Failed data. |
+| random\_string | Random String value created. |
+| sumologic\_collector | Sumo Logic hosted collector. |
+| sumologic\_source | Sumo Logic HTTP source. |