Public AMIs need automatic cleanup in teleport-stage

[camscale]

The overnight release build produces 24 public AMI images each night (4 versions, 6 images per version). Some developer builds are also published on occasion. There is a limit of 200 public AMIs in this account and we hit that limit reasonably quickly, leading to overnight build failures during the "publish AMI" jobs.

We should have an automated clean-up that removes older images so that we always have enough capacity to run the overnight build.

[camscale]

Overnight build failure: https://github.com/gravitational/teleport.e/actions/runs/13615002497/job/38057771875

[fheinecke]

Here are the quick and dirty scripts I use to nuke these every week:

#!/bin/bash

set -o pipefail

echo "Account: $(aws sts get-caller-identity | jq -rc '.Arn')"
read -p "Are you sure (y/n)? " -n 1 -r
echo
if [[ ! ${REPLY} =~ ^[Yy]$ ]]
then
    echo "Did not receive confirmation, aborting."
    exit 1
fi

cleanup-region() {
    region="$1"
    echo "Purging images for ${region}"
    for image in $(aws ec2 describe-images --owners self --region "${region}" | jq -rc '.Images[] | select(.Name | contains("teleport-hardened-base-image-") | not) | {"imageId": .ImageId, "snapshotId": (.BlockDeviceMappings | first | .Ebs.SnapshotId)}'); do
        imageId="$(echo "${image}" | jq -rc '.imageId')"
        snapshotId="$(echo "${image}" | jq -rc '.snapshotId')"
        aws ec2 deregister-image --region "${region}" --image-id "${imageId}"
        aws ec2 delete-snapshot --region "${region}" --snapshot-id "${snapshotId}"
    done
    echo "Region ${region} complete"
}


for region in $(aws account list-regions --region-opt-status-contains ENABLED ENABLING ENABLED_BY_DEFAULT DISABLING | jq -rc '.Regions[].RegionName'); do
    cleanup-region "${region}" &
done

wait < <(jobs -p)

Related: gravitational/shared-workflows#214

[camscale]

This has been completed in https://github.com/gravitational/teleport.e/pull/6146.

The last run was 2 days ago, and it ran successfully. Checking the AMI images, the oldest images in teleport-stage are two days old.