Skip to content

Commit f8f3f7d

Browse files
stoopidJSONstoopidJSON
authored
Merge pull request #98 from CodeForBaltimore/infrastructure
Infrastructure
2 parents 66cfe5d + a94315b commit f8f3f7d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

76 files changed

+2550
-29
lines changed

‎.dockerignore‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Logs
2+
logs
3+
*.log
4+
npm-debug.log*
5+
yarn-debug.log*
6+
yarn-error.log*
7+
lerna-debug.log*
8+
9+
# Diagnostic reports (https://nodejs.org/api/report.html)
10+
report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
11+
12+
# Runtime data
13+
pids
14+
*.pid
15+
*.seed
16+
*.pid.lock
17+
18+
# Directory for instrumented libs generated by jscoverage/JSCover
19+
lib-cov
20+
21+
# Coverage directory used by tools like istanbul
22+
coverage
23+
*.lcov
24+
25+
# nyc test coverage
26+
.nyc_output
27+
28+
# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files)
29+
.grunt
30+
31+
# Bower dependency directory (https://bower.io/)
32+
bower_components
33+
34+
# node-waf configuration
35+
.lock-wscript
36+
37+
# Compiled binary addons (https://nodejs.org/api/addons.html)
38+
build/Release
39+
40+
# Dependency directories
41+
node_modules/
42+
jspm_packages/
43+
44+
# TypeScript v1 declaration files
45+
typings/
46+
47+
# TypeScript cache
48+
*.tsbuildinfo
49+
50+
# Optional npm cache directory
51+
.npm
52+
53+
# Optional eslint cache
54+
.eslintcache
55+
56+
# Microbundle cache
57+
.rpt2_cache/
58+
.rts2_cache_cjs/
59+
.rts2_cache_es/
60+
.rts2_cache_umd/
61+
62+
# Optional REPL history
63+
.node_repl_history
64+
65+
# Output of 'npm pack'
66+
*.tgz
67+
68+
# Yarn Integrity file
69+
.yarn-integrity
70+
71+
# dotenv environment variables file
72+
.env
73+
.env.test
74+
75+
# parcel-bundler cache (https://parceljs.org/)
76+
.cache
77+
78+
# Next.js build output
79+
.next
80+
81+
# Nuxt.js build / generate output
82+
.nuxt
83+
dist
84+
85+
# Gatsby files
86+
.cache/
87+
# Comment in the public line in if your project uses Gatsby and not Next.js
88+
# https://nextjs.org/blog/next-9-1#public-directory-support
89+
# public
90+
91+
# vuepress build output
92+
.vuepress/dist
93+
94+
# Serverless directories
95+
.serverless/
96+
97+
# FuseBox cache
98+
.fusebox/
99+
100+
# DynamoDB Local files
101+
.dynamodb/
102+
103+
# TernJS port file
104+
.tern-port
105+
106+
# Stores VSCode versions used for testing VSCode extensions
107+
.vscode-test
108+
109+
# Mac stupid DS_Store files
110+
*.DS_Store
111+
112+
# terraform
113+
terraform/
114+
.idea
115+
.terraform
116+
*.plan
117+
*.out
118+
119+
# Docker
120+
Dockerfile
121+
docker-compose.yml
122+
docker/
123+
124+
# docs
125+
docs/

‎.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,3 +108,9 @@ dist
108108

109109
# Mac stupid DS_Store files
110110
*.DS_Store
111+
112+
# terraform
113+
.idea
114+
.terraform
115+
*.plan
116+
*.out

‎.pre-commit-config.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
- repo: git://github.com/antonbabenko/pre-commit-terraform
2+
rev: v1.27.0 # Get the latest from: https://github.com/antonbabenko/pre-commit-terraform/releases
3+
hooks:
4+
- id: terraform_fmt
5+
- id: terraform_docs

‎bin/README.md‎

Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
# Deployment Scripts
2+
3+
## Overview
4+
5+
The following scripts are intended to be used for the deployment of the application infrastructure. They have been developed with ease-of-use in mind and, as such, potential input to the script runtime has been significantly restricted.
6+
7+
The scripts in this repository can be divided into two categories: **Invoked Scripts** which represent scripts that a developer would call to launch and administer the application infrastructure, and **Utility Scripts** which are leveraged by the Invoked Scripts to perform common tasks as needed.
8+
9+
## Invoked Scripts
10+
11+
The following scripts are used to control the lifecycle of the application infrastructure. For detailed information about how to launch the infrastructure, please see the [Deployment Documentation](../DEPLOYING.md)
12+
13+
**All invoked scripts should be run from the root of this repository.**
14+
15+
### deploy
16+
17+
##### Usage
18+
19+
```bin/deploy.sh arn:aws:iam::<account number>:mfa/<username>```
20+
21+
The addition of an MFA resource is optional and only required if you are using MFA on your AWS account. If this value is not included, the script will not attempt to generate multi-factor credentials.
22+
23+
##### Description
24+
The deploy script is the main driver for deployment. The usage of the script is described in [DEPLOYING.md](../DEPLOYING.md) and documentation on the individual steps performed in the script are detailed inline as comments.
25+
26+
The most common usage scenario for the deploy script is to perform a full standup of the application infrastructure when newly launching the application or modifying the terraform component(s) that comprise the application runtime environment.
27+
28+
The deploy script completes the following steps:
29+
- Create the Build Agent Docker image (See the [Docker Documentation](../docker/README.md) for more information)
30+
- Ensure that the AWS is configured correctly and attempt an MFA sign-in if the user has passed an MFA device in.
31+
- Builds the API code.
32+
- Creates the Terraform Bucket to hold infrastructure state. (Note: If the bucket exists already, this step does nothing.)
33+
- Runs `terraform apply` to generate the application infrastructure
34+
- Logs into the Amazon Container Registry
35+
- Builds and pushes Docker image for the bmore-responsive api.
36+
- Outputs the S3 Bucket and API addresses for use in interfacing with the APIs
37+
- Cleans up Docker containers that were created as a result of the build process
38+
39+
In the event that any of the above steps return a nonzero error code, the deploy script halts.
40+
41+
### redeploy
42+
43+
##### Usage
44+
45+
`bin/redeploy.sh`
46+
47+
##### Description
48+
49+
The redeploy script is intended for use when any application code for the API or Docker image is modified. When this happens, developers are able to run this script which performs the following actions:
50+
51+
- Rebuilds the Docker Build Agent
52+
- Ensures that AWS is configured correctly
53+
- Builds the API
54+
- Logs into the Amazon Container Registry
55+
- Builds and pushes Docker images for the bmore-responsive api.
56+
- Triggers a container redeployment through the AWS CLI
57+
- Cleans up Docker containers that were created as a result of the build process
58+
59+
Once the redeploy script is run, the bmore-responsive containers will go through the standard AWS rolling redeployment process. In this process, existing containers are kept running while new ones are created and registered as healthy. If the new containers are successfully marked healthy, old containers are drained and removed. This process takes about 5 minutes on average.
60+
61+
The redeploy script is only intended for the redeployment of API code. In the event of a change to any terraform component(s), the `deploy.sh` script should be used to ensure that changes are applied correctly and propagated to all infrastructure components.
62+
63+
**Please note** that the redeploy script does not attempt to perform the MFA login again. It assumes that you have a valid AWS session available when trying to use the script. In the event that this is not the case, the `bin/mfa.sh` script has been supplied to allow multi-factor login.
64+
65+
### cleanup
66+
67+
##### Usage
68+
69+
##### Description
70+
71+
The cleanup script is intended to be used when the application infrastructure is no longer desired. The script will destroy all terraform resources, clear the S3 buckets, and delete the terraform state bucket. **Running this script to completion will render your infrastructure unrecoverable without a full run of `deploy.sh`**
72+
73+
The cleanup script performs the following steps:
74+
75+
- Rebuilds the Docker Build Agent
76+
- Ensures that AWS is configured correctly
77+
- Runs `terraform destroy` **Note:** The developer running this script will be prompted to accept the destruction of the environment.
78+
- Empties the terraform state bucket in use by the infrastructure
79+
- Deletes the terraform state bucket in use by the infrastructure
80+
- Cleans up Docker containers that were created as a result of the build process
81+
82+
### mfa
83+
84+
##### Usage
85+
86+
```bin/mfa.sh arn:aws:iam::<account number>:mfa/<username>```
87+
88+
The addition of an MFA resource is required. If this value is not included, the script will not do anything.
89+
90+
##### Description
91+
92+
This script is a simple wrapper to enable developers to log into AWS MFA for use in the `redeploy.sh` and `cleanup.sh` scripts. It should only be used in the event that a user has not run the `deploy.sh` script recently (or at all) or if their AWS credentials have expired after the given period (36 hours). The script performs the following actions:
93+
94+
- Rebuilds the Docker Build Agent
95+
- Ensures that AWS is configured correctly
96+
- Attempts AWS MFA sign in with the given user credentials
97+
- Cleans up Docker containers that were created as a result of the build process
98+
99+
### Notes
100+
101+
- Every script used for major operations rebuilds the Docker Build Agent. This is to ensure that the agent is at its latest state if changes have occurred between the latest container build and the latest source code updates.
102+
103+
## Utility Scripts
104+
105+
Utility scripts are simple shell scripts designed to be run by the Docker Build Agent when performing application infrastructure administration tasks. They are largely simple scripts designed to strip some complexity from the Invoked Scripts and make them more readable.
106+
107+
**Note:** Most of these scripts require an active AWS session in order to run correctly. Please view the documentation above for `mfa.sh` or `deploy.sh` for more information.
108+
109+
### apply
110+
111+
##### Usage
112+
113+
`apply <Terraform Component> <Extra Args>`
114+
115+
- Terraform Component is the name of the terraform component being deployed. The script will cd to that component's folder and run all build actions from there.
116+
- Extra Args are any other arguments that are normally passed to `terraform apply`. A complete list can be found at the [Terraform Docs](https://www.terraform.io/docs/commands/apply.html)
117+
118+
##### Description
119+
120+
The apply script is designed to be used when Invoked Scripts require terraform to deploy new resources. Apply performs three actions:
121+
122+
- Enters the correct directory for the specified terraform component
123+
- Initializes terraform
124+
- Runs `terraform apply` with any additional arguments
125+
126+
127+
### create-bucket
128+
129+
##### Usage
130+
131+
`create-bucket`
132+
133+
##### Description
134+
135+
This script creates the S3 bucket required for maintaining terraform state. Currently, the target bucket name is hard-coded to `cfb-healthcare-rollcall-terraform-state`. The bucket is created in the `us-east-1` region by default.
136+
137+
### destroy
138+
139+
##### Usage
140+
141+
`destroy <Terraform Component> <Extra Args>`
142+
143+
- Terraform Component is the name of the terraform component being deployed. The script will cd to that component's folder and run all build actions from there.
144+
- Extra Args are any other arguments that are normally passed to `terraform destroy`. A complete list can be found at the [Terraform Docs](https://www.terraform.io/docs/commands/destroy.html)
145+
146+
##### Description
147+
148+
The destroy script is designed to be used when Invoked Scripts require terraform to destroy resources. The script performs three actions:
149+
150+
- Enters the correct directory for the specified terraform component
151+
- Initializes terraform
152+
- Runs `terraform destroy` with any additional arguments
153+
154+
**Note:** Unless `--force` is specified as an extra argument, invoking the `destroy` script will prompt the user for confirmation before performing any actions.
155+
156+
### ecr-login
157+
158+
##### Usage
159+
160+
`ecr-login`
161+
162+
This script is commonly used, however, to invoke a docker login command as such:
163+
164+
`$(ecr-login)`
165+
166+
##### Description
167+
168+
The `ecr-login` script is a simple helper script to wrap the action of logging into the Amazon Elastic Container Registry for your AWS account. This is a necessary step before the execution of any `docker push` activities to the remote AWS Container Registry.
169+
170+
### npm-build
171+
172+
##### Usage
173+
174+
`npm-build `
175+
176+
##### Description
177+
178+
- installs dependencies with `npm install`
179+
- tests code with 'npm test'
180+
- lints code with 'npm lint'
181+
182+
### output
183+
184+
##### Usage
185+
186+
`output <Terraform Component> <Extra Args>`
187+
188+
- Terraform Component is the name of the terraform component being deployed. The script will cd to that component's folder and run all build actions from there.
189+
- Extra Args are any other arguments that are normally passed to `terraform output`. A complete list can be found at the [Terraform Docs](https://www.terraform.io/docs/commands/output.html)
190+
191+
##### Description
192+
193+
The output script is designed to be used when Invoked Scripts require access to any of the outputs from the terraform component. The script performs three actions:
194+
195+
- Enters the correct directory for the specified terraform component
196+
- Initializes terraform
197+
- Runs `terraform output` with any additional arguments
198+
199+
This script is primarily used to pass information about terraform-created resources to other commands in subsequent steps of Invoked Scripts. Additionally, it is used to output information to developers and end-users of the deployment script(s).

‎bin/cleanup.sh‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
#!/bin/bash
2+
3+
# Exit on all nonzero exit codes.
4+
set -e
5+
6+
# Create the builder agent to help us with the rest of the build process
7+
# We run this on all scripts to make sure that the builder is always up-to-date.
8+
docker build -f docker/Dockerfile-Builder -t cfb-build-agent .
9+
10+
# Set AWS Profile to default
11+
export AWS_PROFILE="default"
12+
13+
# Terraform Destroy all of our resources
14+
# We still want to prompt the user for confirmation, especially with an action like this.
15+
docker run -it -v $(pwd):/app/ -v $(pwd)/docker/aws/:/root/.aws/ -e AWS_PROFILE=$AWS_PROFILE cfb-build-agent destroy full-cluster
16+
17+
# Empty out the Terraform State Bucket
18+
docker run -it -v $(pwd):/app/ -v $(pwd)/docker/aws/:/root/.aws/ -e AWS_PROFILE=$AWS_PROFILE cfb-build-agent aws s3 rm s3://cfb-healthcare-rollcall-terraform-state --recursive
19+
20+
# Delete the Terraform State Bucket
21+
docker run -it -v $(pwd):/app/ -v $(pwd)/docker/aws/:/root/.aws/ -e AWS_PROFILE=$AWS_PROFILE cfb-build-agent aws s3api delete-bucket --bucket cfb-healthcare-rollcall-terraform-state --region us-east-1
22+
23+
# Clean up the stopped build agent containers
24+
docker rm $(docker ps -a -q --filter ancestor=cfb-build-agent) > /dev/null 2>&1

0 commit comments

Comments
 (0)