Clean Up AWS Resources After Cluster Deletion

[vivekr-splunk]

Clean Up AWS Resources After Cluster Deletion

Description

This PR introduces automation to clean up any remaining AWS resources, specifically Security Groups and OIDC IDs, after a Kubernetes cluster deletion. The objective is to ensure no orphaned resources are left behind once the pipeline completes, preventing unnecessary resource usage and avoiding potential security risks associated with leftover configurations.

Changes

• Implemented logic to delete AWS Security Groups associated with the cluster after deletion.
• Added cleanup for OIDC ID to remove any lingering identity provider configurations post-cluster deletion.
• Ensured the cleanup process runs at the end of the pipeline to capture any residual resources after all steps have completed.

Why This Is Needed

Leaving behind AWS Security Groups and OIDC configurations can lead to:

• Increased Costs: Unused resources incur costs if not removed.
• Security Risks: Unused Security Groups and identity configurations may expose the account to unauthorized access.
• Resource Clutter: Keeping a clean environment is crucial for maintaining accurate and effective resource management.

Testing

• Verified that Security Groups are correctly identified and deleted after cluster deletion.
• Tested OIDC ID removal in different scenarios to ensure compatibility with existing workflows.
• Confirmed no resources remain after pipeline completion.

Additional Notes

Please review the resource deletion logic to ensure it aligns with existing resource tagging conventions and does not inadvertently delete in-use resources in shared environments.

---

This PR will help maintain a clean AWS environment and improve resource efficiency in our CI/CD pipeline.

[akondur]

Looks good

[vivekr-splunk]

I think security group need to be deleted later once cluster gets cleared ,i will test it out

…

On Tue, Nov 12, 2024 at 9:54 AM Arjun Kondur ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In test/deploy-eks-cluster.sh <#1396 (comment)> : > @@ -21,6 +21,22 @@ if [[ -z "${EKS_CLUSTER_K8_VERSION}" ]]; then fi function deleteCluster() { + echo "Cleanup role, security-group, open-id ${TEST_CLUSTER_NAME}" + account_id=$(aws sts get-caller-identity --query "Account" --output text) + rolename=$(echo ${TEST_CLUSTER_NAME} | awk -F- '{print "EBS_" $(NF-1) "_" $(NF)}') + role_attached_policies=$(aws iam list-attached-role-policies --role-name $rolename --query 'AttachedPolicies[*].PolicyArn' --output text) + for policy_arn in ${role_attached_policies}; + do + aws iam detach-role-policy --role-name ${rolename} --policy-arn ${policy_arn} + done + + aws iam delete-role --role-name ${rolename} + oidc_id=$(aws eks describe-cluster --name ${TEST_CLUSTER_NAME} --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5) + aws iam delete-open-id-connect-provider --open-id-connect-provider-arn arn:aws:iam::${account_id}:oidc-provider/${oidc_id} + security_group_id=$(aws eks describe-cluster --name ${TEST_CLUSTER_NAME} --query "cluster.resourcesVpcConfig.securityGroupIds[0]" --output text) + aws ec2 delete-security-group --group-id ${security_group_id} Looks like security groups also have dependent objects: An error occurred (DependencyViolation) when calling the DeleteSecurityGroup operation: resource sg-0158103d0e31e7ef1 has a dependent object — Reply to this email directly, view it on GitHub <#1396 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWRQER6FAFPXUDT5DHDQNQL2AI6FBAVCNFSM6AAAAABRN67QWWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDIMZQGMZDGNBXGE> . You are receiving this because you were assigned.Message ID: ***@***.***>

[akondur]

I tried deleting it post cluster deletion as well and it fails: . . . 2024-11-12 16:01:18 [✔] all cluster resources were deleted Cluster eks-sok-smoke-test-cluster-managersecret-109839685 deleted successfully . . Deleting security group An error occurred (DependencyViolation) when calling the DeleteSecurityGroup operation: resource sg-0e17b70c84157efe2 has a dependent object I believe it may have network interfaces linked to it. Thanks and Regards, Arjun Kondur On Tue, Nov 12, 2024 at 12:01 PM vivekr-splunk ***@***.***> wrote:

…

I think security group need to be deleted later once cluster gets cleared ,i will test it out On Tue, Nov 12, 2024 at 9:54 AM Arjun Kondur ***@***.***> wrote: > ***@***.**** commented on this pull request. > ------------------------------ > > In test/deploy-eks-cluster.sh > < #1396 (comment)> > : > > > @@ -21,6 +21,22 @@ if [[ -z "${EKS_CLUSTER_K8_VERSION}" ]]; then > fi > > function deleteCluster() { > + echo "Cleanup role, security-group, open-id ${TEST_CLUSTER_NAME}" > + account_id=$(aws sts get-caller-identity --query "Account" --output text) > + rolename=$(echo ${TEST_CLUSTER_NAME} | awk -F- '{print "EBS_" $(NF-1) "_" $(NF)}') > + role_attached_policies=$(aws iam list-attached-role-policies --role-name $rolename --query 'AttachedPolicies[*].PolicyArn' --output text) > + for policy_arn in ${role_attached_policies}; > + do > + aws iam detach-role-policy --role-name ${rolename} --policy-arn ${policy_arn} > + done > + > + aws iam delete-role --role-name ${rolename} > + oidc_id=$(aws eks describe-cluster --name ${TEST_CLUSTER_NAME} --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5) > + aws iam delete-open-id-connect-provider --open-id-connect-provider-arn arn:aws:iam::${account_id}:oidc-provider/${oidc_id} > + security_group_id=$(aws eks describe-cluster --name ${TEST_CLUSTER_NAME} --query "cluster.resourcesVpcConfig.securityGroupIds[0]" --output text) > + aws ec2 delete-security-group --group-id ${security_group_id} > > Looks like security groups also have dependent objects: > > An error occurred (DependencyViolation) when calling the DeleteSecurityGroup operation: resource sg-0158103d0e31e7ef1 has a dependent object > > — > Reply to this email directly, view it on GitHub > < #1396 (review)>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AWRQER6FAFPXUDT5DHDQNQL2AI6FBAVCNFSM6AAAAABRN67QWWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDIMZQGMZDGNBXGE> > . > You are receiving this because you were assigned.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#1396 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQRH3Z2DX47GXYKYATQLGF32AI66RAVCNFSM6AAAAABRN67QWWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZRGIYTSOJQGU> . You are receiving this because your review was requested.Message ID: ***@***.***>