updates

Author: ford-at-aws

.tools/test/stacks/nuke-2/typescript/README.md

@@ -0,0 +1,113 @@
+
+# aws-nuke for Weathertop
+
+AWS Nuke is an open source tool created by [ekristen](https://github.com/ekristen/aws-nuke).
+
+It searches for deleteable resources in the provided AWS account and deletes those which are not considered "Default" or "AWS-Managed", taking your account back to Day 1 with few exceptions.
+
+
+## ⚠ Important
+This is a very destructive tool! It should not be deployed without fully understanding the impact it will have on your AWS accounts.
+Please use caution and configure this tool to delete unused resources only in your lower test/sandbox environment accounts.
+
+## Overview
+
+The code in this repository helps you set up the following architecture:
+
+![infrastructure-overview](architecture-overview.png)
+
+## Feature Outline
+
+1. **Scheduled Trigger**: Amazon EventBridge invokes AWS Step Functions daily.
+2. **Regional Scalability**: Runs AWS CodeBuild projects per region.
+4. **Custom Config**: Pulls resource filters and region targets in [nuke_generic_config.yaml](nuke_generic_config.yaml).
+
+## Prerequisites
+
+1. **AWS-Nuke Binary**: Open-source library from [ekristen](https://github.com/ekristen/aws-nuke) staged in S3.  
+2. **AWS Account Alias**: Must exist in the IAM Dashboard for the target  account.
+8. **Network Connectivity**: Ensure VPC allows downloads from GitHub or stage the binary in S3/artifactory for restricted environments.
+
+## Setup and Installation
+
+* Clone the [repo](https://github.com/ekristen/aws-nuke).
+* Determine the `id` of the account to be deployed for nuking.
+* Narrow [filters](nuke_generic_config.yaml) for the resources/accounts you wish to nuke.
+* Deploy the stack using the below command. You can run it in any desired region.
+```sh
+cdk bootstrap && cdk deploy
+```
+
+Note a successful stack creation, e.g.: 
+
+```bash
+ ✅  NukeCleanser
+
+✨  Deployment time: 172.66s
+
+Outputs:
+NukeCleanser.NukeS3BucketValue = nuke-account-cleanser-config-616362312345-us-east-1-c043b470
+Stack ARN:
+arn:aws:cloudformation:us-east-1:123456788985:stack/NukeCleanser/cfhdkiott-acec-11ef-ba2e-4555c1356d07
+```
+
+Next, run `python upload_job_files.py` to upload two files in this directory: `nuke_config_update.py` and `nuke_generic_config.yaml`.
+
+## When it runs
+* The tool is currently configured to run at a schedule as desired typically off hours 3:00a EST. It can be easily configured with a rate() or cron() expression by editing the cfn template file
+
+* The workflow also sends out a detailed report to an SNS topic with an active email subscription on what resources were deleted after the job is successful for each region which simplifies traversing and parsing the complex logs spit out by the aws-nuke binary. 
+
+* If the workflow is successful, the stack will send out
+  - One email for each of the regions where nuke CodeBuild job was invoked with details of the build execution , the list of resources which was deleted along with the log file path. 
+  - The StepFunctions workflow also sends out another email when the whole Map state process completes successfully. Sample email template given below.
+
+```sh
+      Account Cleansing Process Completed;
+      
+        ------------------------------------------------------------------
+        Summary of the process:
+        ------------------------------------------------------------------
+        DryRunMode                   : true
+        Account ID                   : 123456789012
+        Target Region                : us-west-1
+        Build State                  : JOB SUCCEEDED
+        Build ID                     : AccountNuker-NukeCleanser:4509a9b5
+        CodeBuild Project Name       : AccountNuker-NukeCleanser
+        Process Start Time           : Thu Feb 23 04:05:21 UTC 2023
+        Process End Time             : Thu Feb 23 04:05:54 UTC 2023
+        Log Stream Path              : AccountNuker-NukeCleanser/logPath
+        ------------------------------------------------------------------
+        ################ Nuke Cleanser Logs #################
+      
+        FAILED RESOURCES
+      -------------------------------
+      Total number of Resources that would be removed:
+      3
+      us-west-1 - SQSQueue - https://sqs.us-east-1.amazonaws.com/123456789012/test-nuke-queue - would remove
+      us-west-1 - SNSTopic - TopicARN: arn:aws:sns:us-east-1:123456789012:test-nuke-topic - [TopicARN: "arn:aws:sns:us-east-1:123456789012:test-topic"] - would remove
+      us-west-1 - S3Bucket - s3://test-nuke-bucket-us-west-1 - [CreationDate: "2023-01-25 11:13:14 +0000 UTC", Name: "test-nuke-bucket-us-west-1"] - would remove
+
+```
+
+## Monitoring queries
+
+Use the `aws-cli` to get CloudWatch logs associated with your most recent nuclear activity.
+
+```sh
+# Get the current Unix timestamp
+CURRENT_TIME=$(date +%s000)
+
+# Get the timestamp from 5 minutes ago
+FIVE_MINUTES_AGO=$(($(date +%s) - 300))000
+
+# Filter log events using above date range
+aws logs filter-log-events \
+  --log-group-name AccountNuker-nuke-auto-account-cleanser \
+  --start-time $FIVE_MINUTES_AGO --end-time $CURRENT_TIME \
+  --log-stream-names "10409c89-a90f-4af7-9642-0df9bc9f0855" \
+  --filter-pattern removed \
+  --no-interleaved \
+  --output text \
+  --limit 5
+```

.tools/test/stacks/nuke-2/typescript/account_nuker.ts

@@ -0,0 +1,87 @@
+import * as cdk from 'aws-cdk-lib';
+import * as events from 'aws-cdk-lib/aws-events';
+import * as targets from 'aws-cdk-lib/aws-events-targets';
+import * as iam from 'aws-cdk-lib/aws-iam';
+import * as s3 from 'aws-cdk-lib/aws-s3';
+import * as s3assets from 'aws-cdk-lib/aws-s3-assets';
+import * as s3deploy from 'aws-cdk-lib/aws-s3-deployment'
+import * as path from 'path';
+import * as lambda from "aws-cdk-lib/aws-lambda";
+import * as python from "@aws-cdk/aws-lambda-python-alpha";
+import * as fs from 'fs';
+
+export interface NukeStackProps extends cdk.StackProps {
+  awsNukeDryRunFlag?: string;
+  awsNukeVersion?: string;
+  owner?: string;
+}
+
+class NukeStack extends cdk.Stack {
+  constructor(scope: cdk.App, id: string, props: NukeStackProps) {
+    super(scope, id, props);
+
+    // Applying default props
+    props = {
+      ...props,
+      awsNukeDryRunFlag: props.awsNukeDryRunFlag ?? 'true',
+      awsNukeVersion: props.awsNukeVersion ?? '2.21.2',
+      owner: props.owner ?? 'OpsAdmin',
+    };
+
+    // S3 Bucket for storing AWS Nuke binary and configuration
+    const nukeS3Bucket = new s3.Bucket(this, 'NukeS3Bucket', {
+      bucketName: `nuke-config-bucket-${this.account}-${this.region}`,
+      removalPolicy: cdk.RemovalPolicy.DESTROY,
+      autoDeleteObjects: true,
+      blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
+    });
+
+    const awsNukeConfig = new s3deploy.BucketDeployment(this, 'DeployFile', {
+      sources: [s3deploy.Source.data('nuke_generic_config.yaml', fs.readFileSync('nuke_generic_config.yaml', 'utf-8'))],
+      destinationBucket: nukeS3Bucket,
+      destinationKeyPrefix: 'nuke_generic_config.yaml'
+    })
+  
+    // AWS Lambda Function
+    const nukeLambdaRole = new iam.Role(this, 'NukeLambdaRole', {
+      assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'),
+      managedPolicies: [
+        iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'),
+        iam.ManagedPolicy.fromAwsManagedPolicyName('AmazonS3ReadOnlyAccess'),
+      ],
+    });
+
+    const nukeLambda = new lambda.Function(this, 'NukeLambda', {
+      runtime: lambda.Runtime.PYTHON_3_9,
+      handler: 'index.handler',
+      code: lambda.Code.fromAsset(path.join(__dirname, 'lambda')),
+      role: nukeLambdaRole,
+      timeout: cdk.Duration.minutes(15),
+      environment: {
+        AWS_NUKE_DRY_RUN: props.awsNukeDryRunFlag ?? 'true',
+        AWS_NUKE_VERSION: props.awsNukeVersion ?? '2.21.2',
+        NUKE_S3_BUCKET: nukeS3Bucket.bucketName,
+        NUKE_CONFIG_KEY: cdk.Fn.select(0, awsNukeConfig.objectKeys),
+      },
+    });
+
+    // Grant permissions for the Lambda function to access the S3 bucket
+    nukeS3Bucket.grantRead(nukeLambda);
+
+    // EventBridge Rule
+    const eventBridgeRule = new events.Rule(this, 'NukeScheduleRule', {
+      schedule: events.Schedule.cron({ minute: '0', hour: '7', // Change the schedule as needed
+      }),
+    });
+
+    eventBridgeRule.addTarget(new targets.LambdaFunction(nukeLambda));
+  }
+}
+
+const app = new cdk.App();
+new NukeStack(app, 'NukeStack', {
+  env: {
+    account: process.env.CDK_DEFAULT_ACCOUNT,
+    region: process.env.CDK_DEFAULT_REGION,
+  },
+});

.tools/test/stacks/nuke-2/typescript/cdk.json

@@ -0,0 +1,81 @@
+{
+  "app": "npx ts-node --prefer-ts-exts account_nuker.ts",
+  "watch": {
+    "include": [
+      "**"
+    ],
+    "exclude": [
+      "README.md",
+      "cdk*.json",
+      "**/*.d.ts",
+      "**/*.js",
+      "tsconfig.json",
+      "package*.json",
+      "yarn.lock",
+      "node_modules",
+      "test"
+    ]
+  },
+  "context": {
+    "@aws-cdk/aws-lambda:recognizeLayerVersion": true,
+    "@aws-cdk/core:checkSecretUsage": true,
+    "@aws-cdk/core:target-partitions": [
+      "aws",
+      "aws-cn"
+    ],
+    "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
+    "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
+    "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
+    "@aws-cdk/aws-iam:minimizePolicies": true,
+    "@aws-cdk/core:validateSnapshotRemovalPolicy": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
+    "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
+    "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
+    "@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
+    "@aws-cdk/core:enablePartitionLiterals": true,
+    "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
+    "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
+    "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
+    "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
+    "@aws-cdk/aws-route53-patters:useCertificate": true,
+    "@aws-cdk/customresources:installLatestAwsSdkDefault": false,
+    "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
+    "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
+    "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
+    "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
+    "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
+    "@aws-cdk/aws-redshift:columnId": true,
+    "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
+    "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
+    "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
+    "@aws-cdk/aws-kms:aliasNameRef": true,
+    "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
+    "@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
+    "@aws-cdk/aws-efs:denyAnonymousAccess": true,
+    "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
+    "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
+    "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
+    "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
+    "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true,
+    "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true,
+    "@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true,
+    "@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true,
+    "@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true,
+    "@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true,
+    "@aws-cdk/aws-eks:nodegroupNameAttribute": true,
+    "@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true,
+    "@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true,
+    "@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false,
+    "@aws-cdk/aws-s3:keepNotificationInImportedBucket": false,
+    "@aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions": true,
+    "@aws-cdk/aws-dynamodb:resourcePolicyPerReplica": true,
+    "@aws-cdk/aws-ec2:ec2SumTImeoutEnabled": true,
+    "@aws-cdk/aws-appsync:appSyncGraphQLAPIScopeLambdaPermission": true,
+    "@aws-cdk/aws-rds:setCorrectValueForDatabaseInstanceReadReplicaInstanceResourceId": true,
+    "@aws-cdk/core:cfnIncludeRejectComplexResourceUpdateCreatePolicyIntrinsics": true,
+    "@aws-cdk/aws-lambda-nodejs:sdkV3ExcludeSmithyPackages": true,
+    "@aws-cdk/aws-stepfunctions-tasks:fixRunEcsTaskPolicy": true,
+    "cdk-migrate": true
+  }
+}

.tools/test/stacks/nuke-2/typescript/create_account_alias.py

@@ -0,0 +1,96 @@
+# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
+"""
+This module is used to create an AWS account alias, which is required by the deploy.py script.
+
+It provides a function to create an account alias using the AWS CLI, as this specific
+operation is not supported by the AWS CDK.
+"""
+
+import logging
+import re
+import subprocess
+
+logger = logging.getLogger(__name__)
+
+
+def _is_valid_alias(alias_name: str) -> bool:
+    """
+    Check if the provided alias name is valid according to AWS rules.
+
+    AWS account alias must be unique and must be between 3 and 63 characters long.
+    Valid characters are a-z, 0-9 and '-'.
+
+    Args:
+        alias_name (str): The alias name to validate.
+
+    Returns:
+        bool: True if the alias is valid, False otherwise.
+    """
+    pattern = r"^[a-z0-9](([a-z0-9]|-){0,61}[a-z0-9])?$"
+    return bool(re.match(pattern, alias_name)) and 3 <= len(alias_name) <= 63
+
+
+def _log_aws_cli_version() -> None:
+    """
+    Log the version of the AWS CLI installed on the system.
+    """
+    try:
+        result = subprocess.run(["aws", "--version"], capture_output=True, text=True)
+        logger.info(f"AWS CLI version: {result.stderr.strip()}")
+    except Exception as e:
+        logger.warning(f"Unable to determine AWS CLI version: {str(e)}")
+
+
+def create_account_alias(alias_name: str) -> None:
+    """
+    Create a new account alias with the given name.
+
+    This function exists because the CDK does not support the specific
+    CreateAccountAliases API call. It attempts to create an account alias
+    using the AWS CLI and logs the result.
+
+    If the account alias is created successfully, it logs a success message.
+    If the account alias already exists, it logs a message indicating that.
+    If there is any other error, it logs the error message.
+
+    Args:
+        alias_name (str): The desired name for the account alias.
+    """
+    # Log AWS CLI version when the function is called
+    _log_aws_cli_version()
+
+    if not _is_valid_alias(alias_name):
+        logger.error(
+            f"Invalid alias name '{alias_name}'. It must be between 3 and 63 characters long and contain only lowercase letters, numbers, and hyphens."
+        )
+        return
+
+    command = ["aws", "iam", "create-account-alias", "--account-alias", alias_name]
+
+    try:
+        subprocess.run(
+            command,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            text=True,
+            check=True,
+        )
+        logger.info(f"Account alias '{alias_name}' created successfully.")
+    except subprocess.CalledProcessError as e:
+        if "EntityAlreadyExists" in e.stderr:
+            logger.info(f"Account alias '{alias_name}' already exists.")
+        elif "AccessDenied" in e.stderr:
+            logger.error(
+                f"Access denied when creating account alias '{alias_name}'. Check your AWS credentials and permissions."
+            )
+        elif "ValidationError" in e.stderr:
+            logger.error(
+                f"Validation error when creating account alias '{alias_name}'. The alias might not meet AWS requirements."
+            )
+        else:
+            logger.error(f"Error creating account alias '{alias_name}': {e.stderr}")
+    except Exception as e:
+        logger.error(
+            f"Unexpected error occurred while creating account alias '{alias_name}': {str(e)}"
+        )