feat: Add launch-ec2-runner-with-fallback action

[courtneypacheco]

Integration Testing Evidence

• Link to pull request(s):

  • [DO NOT MERGE] [TESTING] chore: Reference EC2 Runner GitHub action instead of custom workflow code instructlab#3200

• Links to passing CI job(s):

  • Running with a spot instance that got reclaimed by AWS: https://github.com/instructlab/instructlab/actions/runs/13655346606
    • Context: The job technically "failed" because AWS reclaimed the instance, but the logic from this GH action itself succeeded
  • Running on a dedicated instance (i.e., try_spot_instance_first=false): https://github.com/instructlab/instructlab/actions/runs/13664883248

Checklist:

• Commit Message Formatting: Commit titles and messages follow guidelines in the conventional commits.
• Changelog updated with breaking and/or notable changes for the next minor release.
• Documentation has been added and/or updated, if applicable.
• Unit tests have been added and/or updated. (If this is not applicable, please provide a justification.)
• Integration testing has been performed, if applicable

Description of this Change

This action is used to launch an EC2 instance (either as a spot instance or a dedicated instance) in a desired availability zone. If no availability, then backup availability zones will be tried until one is successful.

I've tested this action in the instructlab/instructlab repo by creating a draft PR there that modifies the Large E2E job workflow to reference this proposed action. (See link above.)

Design Considerations

I thought about creating a "single" CI action that is capable of both launching and stopping instances. After all, the base machulav/ec2-runner GitHub action that I leverage under the hood can be used to launch and terminate instances. In the case of that particular GitHub action, users can simply set the mode parameter to either start or stop to indicate if they want to launch or terminate an instance.

Ultimately, I decided against implementing a mode in this PR because I felt that going that route would add complexity to the existing code in this PR, increase the testing surface required, etc.. I do feel that adding a mode would certainly be a useful enhancement for this CI action, but I see that as an enhancement and not a necessity for the initial work. I can certainly file a follow-up issue to add this enhancement, though.

Integration Testing Results

Dedicated Instance Testing

To test the dedicated instance logic, I set try_spot_instance_first = false. This variable tells the CI action to not bother trying to launch as a spot instance. (This is useful if you can't afford to have your instance reclaimed by AWS.

Below are screenshots from the output of a successful run where the CI job launches a dedicated EC2 instance. You can see the failures the CI action encounters when trying to launch in an availability zone that lacks capacity for the desired instance type, but that those failures don't abort the workflow

I then confirmed that the instance was actually cleaned up in AWS. (i.e., we don't have any dangling resources)

Spot Instance Testing

To test the spot instance logic, I set try_spot_instance_first = true, which tells the action to try launching an EC2 spot instance before attempting to launch it as a dedicated instance.

Below is a screenshot of the test run. The spot instance was reclaimed by AWS because AWS needed the capacity back, hence the failures:

See the ! symbols next to the E2E steps that indicate the build was aborted. (I forgot to screenshot evidence on the AWS side).

For clarity, the Stop EC2 Runner step failed because the spot instance was deleted (reclaimed) in AWS, and therefore, there was nothing to actually "stop":

[danmcp]

Nice work!

[danmcp]

This is not a request to change this now, but some cases might want the opposite of use spot instances as a fallback. This can be helpful when cost isn't an issue but availability is. So on demand is preferred, but if spot instances are all that's available, they are better than nothing.

[ktdreyer]

Some jq notes that do not block merging this:

You don't have to cat a file into jq, because jq can read files directly.

jq has a "-r" flag to print strings to stdout without quoting, so you don't need to pipe to tr -d '"' at the end.

[ktdreyer]

I think Courtney was going to update those changes I referenced, so I'll rescind my approval so that Mergify does not auto-merge.