multi-region support in ec2-github-runner upstream

[ktdreyer]

machulav/ec2-github-runner#231 adds support for retrying in multiple regions if there are capacity problems. This is similar to what launch-ec2-runner-with-fallback does.

We should evaluate that PR and see how launch-ec2-runner-with-fallback would integrate with it.

[ktdreyer]

The new availability-zones-config parameter in v2.4.1 takes a string of JSON. The JSON data is an array of objects, like:

[{"imageId":"ami-123", "subnetId":"subnet-123", "securityGroupId":"sg-123"},
 {"imageId":"ami-456", "subnetId":"subnet-456", "securityGroupId":"sg-456"}]

Or multi-region:

[{"imageId":"ami-123", "subnetId":"subnet-123", "securityGroupId":"sg-123"},
 {"imageId":"ami-456", "subnetId":"subnet-456", "securityGroupId":"sg-456", "region":"us-west-2"}]

The author of this feature has a wrapper action, like we do:

https://github.com/unvariance/collector/blob/main/.github/actions/aws-runner/action.yml

This action hard-codes values like subnet IDs, security groups, and AMIs. I think we should hardcode these values into this action as well for clarity.

[courtneypacheco]

I don't think we should hardcode these values into our CI actions. I'd prefer to reference variables.

I'm okay if we want to do something like

[{"imageId":"{{ vars.AWS_EC2_AMI_US_EAST_1 }}", "subnetId":"{{ vars.SUBNET_US_EAST_1A }}", "securityGroupId":"{{ vars.SECURITY_GROUP_ID_US_EAST_1 }}"}]

Keep in mind that our CI actions are released as tags (e.g., "v0.2.0") and published on release branches (e.g., "release-v0.2"). If we start changing our subnet IDs or security group IDs in different tags, then a repo pinning to an older version of one of these actions might reference a subnet ID, SG ID, AMI ID, etc. that no longer exists.

[ktdreyer]

Ok. Many people have poweruser access in the instructlab AWS account for simplicity. That leads to a risk that anyone could easily delete resources that anyone else uses, including the resources we depend on for CI.

This pattern evolved from earlier days when we did not have multiple people dedicated to CI and release engineering. Long-term, we can restrict AWS permissions so that we reduce the risk to the CI's AWS resources.

Let's do your suggestion in the meantime.