Skip to content

Commit 601a583

Browse files
exploriginexplorigin
authored
Merge pull request #8 from RSS-Engineering/add-terraform-plan-action
Add terraform plan action and docs
2 parents a44d6ce + 5503c6f commit 601a583

File tree

9 files changed

+312
-17
lines changed

9 files changed

+312
-17
lines changed

‎docs/README.md‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,22 @@
22

33
The purpose of this repo is to provide guidance on terraform patterns and also serve as a landing space for generic, sharable modules.
44

5-
## Setup and Topology
5+
## [Setup and Topology](topology.md)
66

7-
How a project is organized can have a large impact on maintainability as the project grows and updates are necessary. [Read More...](topology.md)
7+
How a project is organized can have a large impact on maintainability as the project grows and updates are necessary.
8+
9+
[Click here for more details...](topology.md)
10+
11+
## [CI/CD Guidance](cicd/README.md)
12+
13+
Terraform manages your infrastructure by doing the best it can to adapt the current state of your infrastructure to the requested state as defined by your Terraform repo. To do this, Terraform creates a _plan_. How this plan is handled in a CI/CD setup is important to maintaining code review accountability and integrity between production deployment and your projects master branch.
14+
15+
[Click here for more details...](cicd/README.md)
816

917
## Reusable Modules
1018

1119
Terraform modules in this repository are designed to be generic and helpful to the majority case of apps.
12-
They can be included in your Terraform code like this (_kms_secrets_ for example):
20+
They can be included in your Terraform code like this ([kms_secrets](modules/kms_secrets.md) for example):
1321

1422
```terraform
1523
module "secrets" {
@@ -29,6 +37,7 @@ module "secrets" {
2937

3038
Terraform good practice is to specify a commit hash when sourcing an external module to prevent module changes from unexpectly breaking your deployment pipeline.
3139

32-
### kms_secrets
40+
---
41+
### [kms_secrets](modules/kms_secrets.md)
3342

34-
[kms_secrets](modules/kms_secrets.md) allows you to store multiple secrets in your repository in encrypted form. This provides secrets that terraform can use without needing them to be stored and managed in a separate secure store such as PasswordSafe, SecretsManager or as an SSM Param.
43+
The [kms_secrets](modules/kms_secrets.md) module allows you to store multiple secrets in your repository in encrypted form. This provides secrets that terraform can use without needing them to be stored and managed in a separate secure store such as PasswordSafe, SecretsManager or as an SSM Param.

‎docs/cicd/README.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Deploying with CI/CD
2+
3+
It is important to consider these CI/CD best practices and how using Terraform interacts with them.
4+
5+
1. All changes should be reviewed and approved prior to deployment.
6+
2. The *master* branch should be strongly tied to what is deployed to production.
7+
8+
We have well-established patterns for code changes to be reviewed/approved via the pull request process.
9+
However, a *terraform plan* exists in a gray area between code and state. It is possible for the plan to
10+
need to change even if the code that drives it does not (for example if another branch is deployed between
11+
the time a plan is created and when it is to be applied).
12+
13+
## Recommended Terraform State Approval Pattern
14+
15+
The recommended way for terraform plans to be approved is to attach them to the pull request so that
16+
everything is considered at the same time. You will likely need to use branch protection rules in
17+
order to ensure that the plan attached accurately reflects the state of the infrastructure.
18+
19+
This method is geared toward [Github Actions](cicd/deploying_with_github_actions.md). If you need a more
20+
complicated deployment workflow or there are many competing feature branches in your project, you should
21+
consider [CircleCI](cicd/deploying_with_circleci.md).
22+
23+
NOTE: [CircleCI](cicd/deploying_with_circleci.md) is the technically superior option but is quite a bit more expensive than [Github Actions](cicd/deploying_with_github_actions.md).
24+
New projects should start using [Github Actions](cicd/deploying_with_github_actions.md) and only consider [CircleCI](cicd/deploying_with_circleci.md) if [Github Actions](cicd/deploying_with_github_actions.md) is not able
25+
to provide the needed consistency.

‎docs/cicd/deploying_with_circleci.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Deploying with CircleCI
2+
3+
Deploying with CircleCI involves 2 workflows.
4+
5+
## PR Testing and Static Analysis
6+
7+
The first will handle all of the automated validation required to merge a pull request.
8+
9+
```mermaid
10+
graph TB
11+
code[PR Created] --> sa[Static Analysis]
12+
code --> ut[Unit Tests]
13+
sa --Succeeds--> testdeployapproval[CircleCI pauses for approval to deploy to test environment]
14+
ut --Pass--> testdeployapproval
15+
testdeployapproval --> testdeploy[CircleCI deploys to test environment]
16+
testdeploy --Success--> acceptance[CircleCI runs acceptance tests]
17+
acceptance --Succeeds--> gmu[PR Merge Unlocked]
18+
```
19+
20+
## Deployment to production
21+
22+
The second will handle deploying to production.
23+
24+
```mermaid
25+
graph TB
26+
prm[PR merged to master branch] --> tfplan[CircleCI generates a Terraform plan]
27+
tfplan --> pause[CircleCI pauses for Plan Approval]
28+
pause --Approved--> apply[CircleCI applies approved Plan]
29+
```
30+
31+
## More details
32+
33+
At this time, CircleCI should be reserved for large or complicated projects with special cases due to budget reasons.

‎docs/cicd/deploying_with_github_actions.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Deploying with Github Actions
2+
3+
The deployment flow for Github Actions is such that when a Pull Request is created/changed an action will generate a Terraform plan and attach it as a comment to the Pull Request. In order to ensure the consistency of the plan, branch protections should be added to the master branch such that a pull request can only be merged against the latest master commit
4+
5+
```yaml
6+
name: Validate Pull Request
7+
8+
on:
9+
pull_request:
10+
11+
jobs:
12+
check-backend:
13+
terraform:
14+
name: "Build Terraform Plan"
15+
runs-on: ubuntu-latest
16+
17+
steps:
18+
- name: Checkout Repository
19+
uses: actions/checkout@v2
20+
21+
- name: Build Terraform Plan
22+
uses: RSS-Engineering/terraform/gh_actions/attach_plan_to_pr@v1
23+
id: plan
24+
env:
25+
ENV: prod
26+
AWS_REGION: "us-west-2"
27+
AWS_ACCESS_KEY_ID: ${{ secrets.PROD_AWS_ACCESS_KEY_ID }}
28+
AWS_SECRET_ACCESS_KEY: ${{ secrets.PROD_AWS_SECRET_ACCESS_KEY }}
29+
with:
30+
terraform_version: 1.0.1
31+
root: infrastructure/environments/prod
32+
text_artifact_name: tf-plan-${{ github.event.after }}.txt
33+
plan_artifact_name: tf-plan-${{ github.event.after }}
34+
35+
- name: Terraform Plan Status
36+
if: steps.plan.outcome == 'failure'
37+
run: exit 1
38+
```

‎docs/index.html‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@
2424
<!-- Docsify v4 -->
2525
<script src="//cdn.jsdelivr.net/npm/docsify@4"></script>
2626
<script src="//cdn.jsdelivr.net/npm/docsify-copy-code/dist/docsify-copy-code.min.js"></script>
27-
<script src="//cdn.jsdelivr.net/npm/docsify-pagination/dist/docsify-pagination.min.js"></script>
27+
<script src="//unpkg.com/mermaid/dist/mermaid.js"></script>
28+
<script src="//unpkg.com/docsify-mermaid@latest/dist/docsify-mermaid.js"></script>
29+
<script>mermaid.initialize({ startOnLoad: true });</script>
2830
</body>
2931
</html>

‎docs/topology.md‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,24 @@
11
# Recommended Infrastructure Topology
22

3-
The recommendations in this repository assume a multiple-account structure where each project/environment has its own provider (AWS) account. In some cases, there may be a master account to hold common resources but projects should try to keep resources as separate as possible for security and asset-management reasons. ([RAX.IO talk](https://web.microsoftstream.com/video/10e4abe9-fcf6-4a01-b1b7-6f4919a0a28b?channelId=3237e715-5c23-4833-a0a5-1690a7437c3a)) ([further reading](https://www.terraform.io/docs/cloud/guides/recommended-practices/part2.html))
3+
The recommendations in this repository assume a multiple-account structure where each project/environment has its own provider (AWS) account. In some cases, there may be a master account to hold common resources but projects should try to keep resources as separate as possible for security and asset-management reasons.
4+
5+
- [RAX.IO talk](https://web.microsoftstream.com/video/10e4abe9-fcf6-4a01-b1b7-6f4919a0a28b?channelId=3237e715-5c23-4833-a0a5-1690a7437c3a)
6+
- [further reading](https://www.terraform.io/docs/cloud/guides/recommended-practices/part2.html)
47

58
## Project Repository Structure
69

710
To achieve code-reuse and environment segregation, place your terraform code in a sub-directory called _infrastructure_ with a structure like:
811

9-
```
10-
- infrastructure
11-
- env
12-
- staging
12+
```text
13+
- infrastructure/
14+
- env/
15+
- staging/
1316
- conf.tf
1417
- main.tf
15-
- production
18+
- production/
1619
- conf.tf
1720
- main.tf
18-
- modules
21+
- modules/
1922
- [project-specific modules]
2023
- main.tf
2124
- variables.tf
@@ -41,7 +44,7 @@ Terraform needs to store the current state of each environment in a place where
4144
### Setting up Remote State
4245

4346
Move the [sample-file](https://github.com/RSS-Engineering/terraform/blob/main/backend_state_init/backend.tf.sample) to terraform file and then remove the sample file.
44-
```bash
47+
```shell
4548
mv backend_state_init/backend.tf.sample backend_state_init/backend.tf
4649
```
4750

@@ -67,9 +70,8 @@ locals {
6770
}
6871
}
6972
```
70-
7173

72-
```bash
74+
```shell
7375
cd backend_state_init/
7476
# Execute the terraform script for setting backend states in s3 and dynamo db.
7577
terraform init

‎gh_actions/attach_plan_to_pr/README.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Terraform Plan action
2+
3+
This action creates a Terraform plan from the current repository. It will output text and can also post a message to a pull request and save the text and data plan files as artifacts.
4+
5+
## Inputs
6+
7+
### terraform_version
8+
9+
**Required** The terraform version to use.
10+
11+
### root
12+
13+
**Required** The root directory of the terraform repo (Passed to -chdir)
14+
15+
### validate_format
16+
17+
Validate the *.tf files format. (default: `true`)
18+
19+
### cache_providers
20+
21+
Should the action try to cache Terraform providers (default: `true`)
22+
23+
### add_plan_to_pr
24+
25+
For Pull Request events, add a plan comment to the current PR (default: `true`)
26+
27+
### text_artifact_name
28+
29+
Save the plan text as an artifact with the provided name.
30+
31+
### plan_artifact_name
32+
33+
Save the plan as an artifact with the provided name
34+
35+
## Outputs
36+
37+
## `text`
38+
39+
The terraform plan in human-readable output
40+
41+
## Example usage
42+
43+
```yaml
44+
- name: Build Terraform Plan
45+
uses: ./.github/actions/attach_plan_to_pr
46+
id: plan
47+
env:
48+
ENV: prod
49+
AWS_DEFAULT_REGION: "us-west-2"
50+
AWS_ACCESS_KEY_ID: ${{ secrets.PROD_AWS_ACCESS_KEY_ID }}
51+
AWS_SECRET_ACCESS_KEY: ${{ secrets.PROD_AWS_SECRET_ACCESS_KEY }}
52+
with:
53+
terraform_version: 1.0.1
54+
root: infrastructure/environments/prod
55+
text_artifact_name: tf-plan-${{ github.event.pull_request.head.sha }}.txt
56+
plan_artifact_name: tf-plan-${{ github.event.pull_request.head.sha }}
57+
```
58+

‎gh_actions/attach_plan_to_pr/action.yml‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# action.yml
2+
name: 'Build Terraform Plan'
3+
description: 'Builds a terraform plan'
4+
author: 'Timothy Farrell'
5+
6+
inputs:
7+
terraform_version:
8+
description: 'Terraform version'
9+
required: true
10+
root:
11+
description: 'Run terraform in this subdirectory. Passed to -chdir'
12+
required: true
13+
validate_format:
14+
description: 'Validate the tf file format'
15+
required: false
16+
default: 'true'
17+
cache_providers:
18+
description: 'Set to "" to not try to cache providers'
19+
required: false
20+
default: 'true'
21+
add_plan_to_pr:
22+
description: 'For Pull Request events, add a plan comment to the current PR'
23+
required: false
24+
default: 'true'
25+
text_artifact_name:
26+
description: 'Save the plan text as an artifact with the provided name'
27+
required: false
28+
default: ''
29+
plan_artifact_name:
30+
description: 'Save the plan as an artifact with the provided name'
31+
required: false
32+
default: ''
33+
outputs:
34+
text:
35+
description: 'Plan text'
36+
value: ${{ steps.plan.outputs.stdout }}
37+
38+
runs:
39+
using: "composite"
40+
steps:
41+
- name: Setup Terraform
42+
uses: hashicorp/setup-terraform@v1
43+
with:
44+
terraform_version: ${{ inputs.terraform_version }}
45+
46+
- name: Hash Lockfile
47+
id: lockfile_path
48+
if: inputs.cache_providers
49+
shell: bash
50+
run: echo "${{ inputs.root }}/.terraform.lock.hcl"
51+
52+
- name: Load cached Terraform Providers
53+
uses: actions/cache@v3
54+
if: inputs.cache_providers && hashFiles(steps.lockfile_path.output.stdout) != ''
55+
with:
56+
path: ${{ inputs.root }}/.terraform
57+
key: tf-providers-${{ hashFiles(steps.lockfile_path.output.stdout) }}
58+
59+
- name: Initialize Terraform
60+
shell: bash
61+
run: terraform -chdir=${{ inputs.root }} init
62+
63+
- name: Validate Terraform
64+
if: inputs.validate_format != 'true'
65+
shell: bash
66+
run: |
67+
terraform fmt -check -recursive -diff infrastructure
68+
terraform -chdir=${{ inputs.root }} validate
69+
70+
- name: Terraform Plan
71+
id: plan
72+
shell: bash
73+
run: terraform -chdir=${{ inputs.root }} plan -out "tf_plan_${{ github.event.after }}" -no-color -input=false
74+
75+
- name: Save Terraform Plan
76+
if: inputs.plan_artifact_name != '' && steps.plan.outcome == 'success'
77+
uses: actions/upload-artifact@v3
78+
with:
79+
name: ${{ inputs.plan_artifact_name }}
80+
path: ${{ inputs.root }}/tf_plan_${{ github.event.after }}
81+
if-no-files-found: error
82+
retention-days: 7
83+
84+
- name: Dump Plan Text to File
85+
if: inputs.text_artifact_name != '' && steps.plan.outcome == 'success'
86+
shell: bash
87+
env:
88+
PLAN: "${{ steps.plan.outputs.stdout }}"
89+
run: echo "${PLAN}" >> tf-plan-${{ github.event.after }}.txt
90+
91+
- name: Save Terraform Plan Text
92+
if: inputs.text_artifact_name != '' && steps.plan.outcome == 'success'
93+
uses: actions/upload-artifact@v3
94+
with:
95+
name: ${{ inputs.text_artifact_name }}
96+
path: ${{ inputs.text_artifact_name }}
97+
if-no-files-found: error
98+
retention-days: 7
99+
100+
- uses: actions/github-script@0.9.0
101+
if: steps.plan.outcome == 'success' && inputs.add_plan_to_pr && github.event_name == 'pull_request'
102+
env:
103+
PLAN: "${{ steps.plan.outputs.stdout }}"
104+
with:
105+
github-token: ${{ github.token }}
106+
script: |
107+
const output = `#### Terraform Plan 📖 \`${{ steps.plan.outcome }}\`
108+
<details><summary>Show Plan</summary>
109+
110+
\`\`\`terraform\n
111+
${process.env.PLAN}
112+
\`\`\`
113+
114+
</details>
115+
116+
*Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`*
117+
`;
118+
github.issues.createComment({
119+
issue_number: context.issue.number,
120+
owner: context.repo.owner,
121+
repo: context.repo.repo,
122+
body: output
123+
})
124+
125+
- name: Terraform Plan Status
126+
if: steps.plan.outcome == 'failure'
127+
shell: bash
128+
run: exit 1

‎modules/lambda-layer-deps/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ No modules.
3131
| Name | Type |
3232
|------|------|
3333
| [aws_lambda_layer_version.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_layer_version) | resource |
34-
| [external_external.build](https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/external) | data source |
34+
| [external_data_source.build](https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/data_source) | data source |
3535

3636
## Inputs
3737

0 commit comments

Comments
 (0)