"Switch on-off" automation proposal

[NivenPrasad]

Overview

What kind of repetitive thing do you have to do often and what is the benefit of automating it?
some HFLA projects use a bastion host for securing access to private resources (like databases or servers that do not face the public internet).

Running this bastion server all the time costs money, even though it's not always being used. This automation would set a timed or trigger-able action to launch and/or destroy one of these bastion servers when needed.

Ownership of Idea

• I will be working on this automation myself
• This is my idea and I need help developing it
• Free for anyone to work on this idea (@jafow @mattyweb can offer guidance!)

Current State

"As-is" most likely something manual but could be partially automated

Future Development

• create a github action that

Action Items/Research

• https://docs.aws.amazon.com/cli/latest/reference/
• https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions

Stakeholders

Impact - who benefits and how?
hfla projects using bastion hosts (ballotnav, 311, others?) will save $$ each month on hosting bill

Anticipated outcomes

Resources/Instructions

Language

Platform

Automation triggers (What starts it? What's it responding to?)

• Time-based (Specify frequency (e.g. 1x/week)
• Event-based (e.g. someone just created a new GitHub on a repo)
  maybe both?
  perhaps an action could run at 11:59PM PST every night and turn the bastion off.
  then a person could launch it when they needed to

Input required (How much manual or custom input is required?)

a person should be able launch a bastion server by some github action -- an issue? a pr? a label? I dunno!

Output

(What's the desired result? What do we not want to see?)

Project size

[jafow]

this is open for grabs I can help guide on it. it might be really fun!

[akibrhast]

Would like to grab this!

[mattyweb]

Adding some background here:

The bastion works great. we use it for secure access to the DB. the client i use (Postico) can be configured to SSH tunnel so it's a seamless process to connect. Occasionally I set up a tunnel from a terminal.

I connect for 2 reasons really.

1. I'm developing/tuning a new query and want to see how it's going to perform in production on what's now approaching 7 million records.
2. I just did a release that includes a schema change and I want to run an alembic migration. Sometimes I need to undo and redo the migration.

The challenge is that the bastion as currently configured is in an auto-scaling group so it's designed to make it impossible to turn off. I can manually reconfigure the auto-scaling group but Terraform will change it back on the next push.

The solution is deceptively simple. all that's needed to fix it is to set the min number of instances to 0 instead of 1 in the auto-scaling group. However, since it's a project referenced by a project referenced by a project and there are no variables to control this that's easier said than done...

[akibrhast]

Possible Resources

https://github.com/hackforla/311-data/tree/dev/server/terraform

[akibrhast]

More Context Regarding this as mentioned by - @jafow

• Bastion costs are between 1-5$ month.
• The bastion runs always because we do not know when someone may want to open a db connection. The goal of this automation is to enable what you describe, on demand connections to db via bastion
• Doesn't have to be github action but the idea is that someone who needs access can launch this bastion without needing aws knowledge or keys etc
• There are surely other ways to solve this. Each will have tradeoffs. If you're into research that may be a good place to begin
• We can allow an automation (eg github actions) to create a bastion without allowing the automation to access the resources the bastion protects
• A more general problem statement might be, how can I enable secure access to resources in a private vpc (virtual private cloud) from the outside world with the lowest cost, lowest barrier to entry (ie one click and done!)

[akibrhast]

A little bit of googling based on what you last mentioned @jafow . I came upon this

The conclusion of that article is basically they have a button that allows one to scale down to 0 or suspend an ASG

So what you are looking for is

1. A github trigger of type workflow-dispatch(a manual trigger)
2. With input 0 or 1
3. Use ubuntu latest
4. download aws cli
5. retrieve relevant aws keys from github secrets
6. Using aws cli and keys from github secrets
7. 0 sets the scale of ASG's to 0, turning it off
8. 1 set's the ASG to it's original initial configuration
9. done

[akibrhast]

After talking to @darpham . It seems like this is about the extent of the workflow file that is going to be needed to control the asg of the server

https://github.com/akibrhast/github-actions-test/blob/master/.github/workflows/bastion_state_manager.yml

[akibrhast]

Unassigning myself from this for the moment since I have not heard a response back on this in the last month and not sure where to go from here.