AWS Resource Management: Cleanup and Destruction Utilities

[jeremymanning]

Summary

Document AWS resource cleanup utilities that should be migrated to proper infrastructure management.

Utility 1: Test Resource Cleanup

From cleanup_test_resources.py - Systematic AWS resource cleanup:

Features:

• NAT gateway deletion with Elastic IP release
• VPC cleanup with all associated resources (subnets, route tables, security groups, IGW)
• Handles resource dependencies correctly
• Cost-aware cleanup to prevent charges

Key Pattern:

def cleanup_resources():
    # Delete NAT gateways first (expensive resources)
    for nat in nats['NatGateways']:
        ec2.delete_nat_gateway(NatGatewayId=nat_id)
        
        # Release associated Elastic IPs
        for addr in nat.get('NatGatewayAddresses', []):
            if 'AllocationId' in addr:
                ec2.release_address(AllocationId=addr['AllocationId'])
    
    # Clean up VPC resources in dependency order
    # subnets -> route tables -> security groups -> IGW -> VPC

Utility 2: EKS Cluster Destruction

From destroy_cluster.py - Complete EKS cluster teardown:

Features:

• Node group deletion with wait conditions
• Cluster deletion with proper timing
• VPC and associated resource cleanup
• IAM role and policy cleanup
• Comprehensive error handling

Key Pattern:

def destroy_cluster(cluster_name, region):
    # 1. Delete node groups first
    for ng_name in nodegroups:
        eks.delete_nodegroup(clusterName=cluster_name, nodegroupName=ng_name)
        waiter.wait()  # Critical: wait for completion
    
    # 2. Delete cluster
    eks.delete_cluster(name=cluster_name)
    waiter.wait()
    
    # 3. Clean up VPC (tagged with cluster name)
    # 4. Clean up IAM roles

Migration Recommendation

These utilities should be moved to scripts/aws/ directory and enhanced:

1. Move to proper location: scripts/aws/cleanup_resources.py and scripts/aws/destroy_cluster.py
2. Add safeguards: Confirmation prompts, dry-run mode
3. Improve error handling: Better error messages and partial failure recovery
4. Add logging: Structured logging for audit trails
5. Configuration: Support for different AWS profiles/regions

Value

• Prevent AWS charges from abandoned test resources
• Provide safe cluster teardown procedures
• Handle complex AWS resource dependencies
• Support infrastructure as code practices

Source: Repository cleanup Issue #72