Skip to content

Commit 5be34c4

Browse files
authored
First pass at a full terraform example (nhs-england-tools#151)
<!-- markdownlint-disable-next-line first-line-heading --> ## Description As a general rule, the repository template should provide functionality that works out of the box. The terraform example is problematic because the options default to running the example, and the example doesn't give any guidance as to how people should set up their own code. This patch removes the example, and replaces it with a more fully explained quick start example in `Scripting_Terraform.md`. It also adds the `TF_ENV` environment variable, as a shorthand for selecting the terraform environment directory under `infrastructure/environments`. ## Type of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply. --> - [ ] Refactoring (non-breaking change) - [x] New feature (non-breaking change which adds functionality) - [x] Breaking change (fix or feature that would change existing functionality) - [ ] Bug fix (non-breaking change which fixes an issue) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. --> - [ ] I am familiar with the [contributing guidelines](../docs/CONTRIBUTING.md) - [ ] I have followed the code style of the project - [ ] I have added tests to cover my changes - [ ] I have updated the documentation accordingly - [ ] This PR is a result of pair or mob programming --- ## Sensitive Information Declaration To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including [PII (Personal Identifiable Information) / PID (Personal Identifiable Data)](https://digital.nhs.uk/data-and-information/keeping-data-safe-and-benefitting-the-public) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter. - [x] I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.
1 parent 5a9f3f2 commit 5be34c4

File tree

8 files changed

+190
-147
lines changed

8 files changed

+190
-147
lines changed

docs/developer-guides/Scripting_Terraform.md

Lines changed: 185 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -62,35 +62,202 @@ Here are some key features built into this repository's Terraform module:
6262

6363
### Quick start
6464

65-
Run the example:
65+
The Repository Template assumes that you will be constructing the bulk of your infrastructure in `infrastructure/modules` as generic deployment configuration, which you will then compose into environment-specific modules, each stored in their own directory under `infrastructure/environments`. Let's create a simple deployable thing, and configure an S3 bucket. We'll make the name of the bucket a variable, so that each environment can have its own.
66+
67+
Open the file `infrastructure/modules/private_s3_bucket/main.tf`, and put this in it:
68+
69+
```terraform
70+
# Define the provider
71+
provider "aws" {
72+
region = "eu-west-2"
73+
}
74+
75+
variable "bucket_name" {
76+
description = "Name of the bucket, which can be different per environment"
77+
}
78+
79+
resource "aws_s3_bucket" "my_bucket" {
80+
bucket = var.bucket_name # Replace with your desired bucket name
81+
acl = "private"
82+
}
83+
```
84+
85+
Note that the variable has been given no value. This is intentional, and allows us to pass the bucket name in as a parameter from the environment.
86+
87+
Now, we're going to define two deployment environments: `dev`, and `test`. Run this:
88+
89+
```bash
90+
mkdir -p infrastructure/environments/{dev,test}
91+
```
92+
93+
It is important that the directory names match your environment names.
94+
95+
Now, let's create the environment definition files. Open `infrastructure/environments/dev/main.tf` and copy in:
96+
97+
```terraform
98+
module "dev_environment" {
99+
source = "../../modules/private_s3_bucket"
100+
bucket_name = "nhse-ee-my-fancy-bucket"
101+
}
102+
```
103+
104+
Some things to note:
105+
106+
- The `source` path is relative to the directory that the `main.tf` file is in. When `terraform` runs, it will `chdir` to that directory first, before doing anything else.
107+
- The `module` name, `"dev_environment"` here, can be anything. Module names are only scoped to the file they're in, so you don't need to follow any particular convention here.
108+
- The `bucket_name` is going to end up as the bucket name in AWS. It wants to be meaningful to you, and you need to pick your own. The framework doesn't constrain your choice, but remember that AWS needs them to be globally unique and if you steal `"nhse-ee-my-fancy-bucket"` then I can't test these docs and then I will be sad.
109+
110+
Let's create our `test` environment now. Open `infrastructure/environments/test/main.tf` and copy in:
111+
112+
```terraform
113+
module "test_environment" {
114+
source = "../../modules/private_s3_bucket"
115+
bucket_name = "nhse-ee-my-fancy-test-bucket"
116+
}
117+
```
118+
119+
We have changed the bucket name here. In this example, I am making no assumptions as to how your AWS accounts are set up. If you intend for your development and test infrastructure to be in the same AWS account (perhaps by necessity, for organisational reasons) and you need to separate them by a naming convention, the framework can support that.
120+
121+
Now we have our modules and our environments configured, we need to initialise each of them. Run these two commands:
122+
123+
```bash
124+
TF_ENV=dev make terraform-init
125+
TF_ENV=test make terraform-init
126+
```
127+
128+
Each invocation will download the `terraform` dependencies we need. The `TF_ENV` name we give to each invocation is the name of the environment, and must match the directory name we chose under `infrastructure/environments` so that `make` gives the right parameters to `terraform`.
129+
130+
We are now ready to try deploying to AWS, from our local environment.
131+
132+
I am going to assume that you have an `~/.aws/credentials` file set up with a separate profile for each environment that you want to use, called `my-test-environment` and `my-dev-environment`. They might have the same credential values in them, in which case `terraform` will create the resources in the same account; or you might have them set up to deploy to different accounts. Either would work.
133+
134+
Run the following:
135+
136+
```shell
137+
TF_ENV=dev AWS_PROFILE=my-dev-environment make terraform-plan
138+
```
139+
140+
If all is working correctly (and you may need to do a round of `aws sso login` first), you should see this output:
141+
142+
```text
143+
144+
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
145+
+ create
146+
147+
Terraform will perform the following actions:
148+
149+
# module.dev_environment.aws_s3_bucket.my_bucket will be created
150+
+ resource "aws_s3_bucket" "my_bucket" {
151+
+ acceleration_status = (known after apply)
152+
+ acl = "private"
153+
+ arn = (known after apply)
154+
+ bucket = "my-dev-bucket"
155+
+ bucket_domain_name = (known after apply)
156+
+ bucket_prefix = (known after apply)
157+
+ bucket_regional_domain_name = (known after apply)
158+
+ force_destroy = false
159+
+ hosted_zone_id = (known after apply)
160+
+ id = (known after apply)
161+
+ object_lock_enabled = (known after apply)
162+
+ policy = (known after apply)
163+
+ region = (known after apply)
164+
+ request_payer = (known after apply)
165+
+ tags_all = (known after apply)
166+
+ website_domain = (known after apply)
167+
+ website_endpoint = (known after apply)
168+
}
169+
170+
Plan: 1 to add, 0 to change, 0 to destroy.
171+
172+
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
173+
174+
Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.
175+
176+
```
177+
178+
No errors found, so we can now create the bucket:
66179

67180
```shell
68-
# AWS console access setup
69-
export AWS_ACCESS_KEY_ID="..."
70-
export AWS_SECRET_ACCESS_KEY="..."
71-
export AWS_SESSION_TOKEN="..."
181+
$ TF_ENV=dev AWS_PROFILE=my-dev-environment make terraform-apply
182+
183+
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
184+
+ create
185+
186+
Terraform will perform the following actions:
187+
188+
# module.dev_environment.aws_s3_bucket.my_bucket will be created
189+
+ resource "aws_s3_bucket" "my_bucket" {
190+
+ acceleration_status = (known after apply)
191+
+ acl = (known after apply)
192+
+ arn = (known after apply)
193+
+ bucket = "nhse-ee-my-dev-bucket"
194+
+ bucket_domain_name = (known after apply)
195+
+ bucket_prefix = (known after apply)
196+
+ bucket_regional_domain_name = (known after apply)
197+
+ force_destroy = false
198+
+ hosted_zone_id = (known after apply)
199+
+ id = (known after apply)
200+
+ object_lock_enabled = (known after apply)
201+
+ policy = (known after apply)
202+
+ region = (known after apply)
203+
+ request_payer = (known after apply)
204+
+ tags_all = (known after apply)
205+
+ website_domain = (known after apply)
206+
+ website_endpoint = (known after apply)
207+
}
208+
209+
Plan: 1 to add, 0 to change, 0 to destroy.
210+
211+
Do you want to perform these actions?
212+
Terraform will perform the actions described above.
213+
Only 'yes' will be accepted to approve.
214+
215+
Enter a value: yes
216+
217+
module.dev_environment.aws_s3_bucket.my_bucket: Creating...
218+
module.dev_environment.aws_s3_bucket.my_bucket: Creation complete after 1s [id=nhse-ee-my-dev-bucket]
219+
220+
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
221+
72222
```
73223

224+
You will notice here that I needed to confirm the action to `terraform` manually. If you don't want to do that, you can pass the `-auto-approve` option to `terraform` like this:
225+
74226
```shell
75-
$ make terraform-example-provision-aws-infrastructure
227+
TF_ENV=dev AWS_PROFILE=my-dev-environment make terraform-apply opts="-auto-approve"
228+
```
76229

77-
Initializing the backend..
78-
...
79-
Plan: 5 to add, 0 to change, 0 to destroy.
80-
Saved the plan to: terraform.tfplan
81-
To perform exactly these actions, run the following command to apply:
82-
terraform apply "terraform.tfplan"
230+
If you check the contents of your AWS account, you should see your new bucket:
231+
232+
```shell
233+
$ aws s3 ls --profile my-dev-environment
83234
...
84-
Apply complete! Resources: 5 added, 0 changed, 0 destroyed.
235+
2024-03-01 16:33:55 nhse-ee-my-dev-bucket
236+
```
85237

86-
$ make terraform-example-destroy-aws-infrastructure
238+
Now I don't want to leave that there, so I will run the corresponding `destroy` command to get rid of it:
87239

88-
...
89-
Plan: 0 to add, 0 to change, 5 to destroy.
90-
...
91-
Apply complete! Resources: 0 added, 0 changed, 5 destroyed.
240+
```shell
241+
$ TF_ENV=dev AWS_PROFILE=my-dev-environment make terraform-destroy opts="-auto-approve"
242+
module.dev_environment.aws_s3_bucket.my_bucket: Refreshing state... [id=nhse-ee-my-dev-bucket]
243+
244+
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
245+
- destroy
246+
247+
Terraform will perform the following actions:
248+
249+
# module.dev_environment.aws_s3_bucket.my_bucket will be destroyed
250+
...(more terraform output not shown because it's boring, but the end result is the bucket going away)
92251
```
93252
253+
To create your `test` environment, you run the same commands with `test` where previously you had `dev`:
254+
255+
```shell
256+
TF_ENV=test AWS_PROFILE=my-test-environment make terraform-apply opts="-auto-approve"
257+
```
258+
259+
To use the same `terraform` files in a GitHub action, see the docs [here](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services).
260+
94261
### Your stack implementation
95262
96263
Always follow [best practices for using Terraform](https://cloud.google.com/docs/terraform/best-practices-for-terraform) while providing infrastructure as code (IaC) for your service.

scripts/terraform/examples/terraform-state-aws-s3/.gitignore

Lines changed: 0 additions & 41 deletions
This file was deleted.

scripts/terraform/examples/terraform-state-aws-s3/main.tf

Lines changed: 0 additions & 46 deletions
This file was deleted.

scripts/terraform/examples/terraform-state-aws-s3/provider.tf

Lines changed: 0 additions & 3 deletions
This file was deleted.

scripts/terraform/examples/terraform-state-aws-s3/variables.tf

Lines changed: 0 additions & 9 deletions
This file was deleted.

scripts/terraform/examples/terraform-state-aws-s3/versions.tf

Lines changed: 0 additions & 8 deletions
This file was deleted.

scripts/terraform/terraform.mk

Lines changed: 5 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,11 @@
44
# Custom implementation - implementation of a make target should not exceed 5 lines of effective code.
55
# In most cases there should be no need to modify the existing make targets.
66

7+
TF_ENV ?= dev
8+
STACK ?= ${stack}
9+
TERRAFORM_STACK ?= $(or ${STACK}, infrastructure/environments/${TF_ENV})
10+
dir ?= ${TERRAFORM_STACK}
11+
712
terraform-init: # Initialise Terraform - optional: terraform_dir|dir=[path to a directory where the command will be executed, relative to the project's top-level directory, default is one of the module variables or the example directory, if not set], terraform_opts|opts=[options to pass to the Terraform init command, default is none/empty] @Development
813
make _terraform cmd="init" \
914
dir=$(or ${terraform_dir}, ${dir}) \
@@ -41,8 +46,6 @@ clean:: # Remove Terraform files (terraform) - optional: terraform_dir|dir=[path
4146
opts=$(or ${terraform_opts}, ${opts})
4247

4348
_terraform: # Terraform command wrapper - mandatory: cmd=[command to execute]; optional: dir=[path to a directory where the command will be executed, relative to the project's top-level directory, default is one of the module variables or the example directory, if not set], opts=[options to pass to the Terraform command, default is none/empty]
44-
# 'TERRAFORM_STACK' is passed to the functions as environment variable
45-
TERRAFORM_STACK=$(or ${TERRAFORM_STACK}, $(or ${terraform_stack}, $(or ${STACK}, $(or ${stack}, scripts/terraform/examples/terraform-state-aws-s3))))
4649
dir=$(or ${dir}, ${TERRAFORM_STACK})
4750
source scripts/terraform/terraform.lib.sh
4851
terraform-${cmd} # 'dir' and 'opts' are accessible by the function as environment variables, if set
@@ -55,23 +58,6 @@ terraform-shellscript-lint: # Lint all Terraform module shell scripts @Quality
5558
file=$${file} scripts/shellscript-linter.sh
5659
done
5760

58-
# ==============================================================================
59-
# Module tests and examples - please DO NOT edit this section!
60-
61-
terraform-example-provision-aws-infrastructure: # Provision example of AWS infrastructure @ExamplesAndTests
62-
make terraform-init
63-
make terraform-plan opts="-out=terraform.tfplan"
64-
make terraform-apply opts="-auto-approve terraform.tfplan"
65-
66-
terraform-example-destroy-aws-infrastructure: # Destroy example of AWS infrastructure @ExamplesAndTests
67-
make terraform-destroy opts="-auto-approve"
68-
69-
terraform-example-clean: # Remove Terraform example files @ExamplesAndTests
70-
dir=$(or ${dir}, ${TERRAFORM_STACK})
71-
source scripts/terraform/terraform.lib.sh
72-
terraform-clean
73-
rm -f ${TERRAFORM_STACK}/.terraform.lock.hcl
74-
7561
# ==============================================================================
7662
# Configuration - please DO NOT edit this section!
7763

@@ -85,9 +71,6 @@ ${VERBOSE}.SILENT: \
8571
clean \
8672
terraform-apply \
8773
terraform-destroy \
88-
terraform-example-clean \
89-
terraform-example-destroy-aws-infrastructure \
90-
terraform-example-provision-aws-infrastructure \
9174
terraform-fmt \
9275
terraform-init \
9376
terraform-install \

0 commit comments

Comments
 (0)