Skip to content

Commit b0e5d5a

Browse files
Jamie-BitFlightaknysh
Jamie-BitFlight
authored and
aknysh
committed
Added an S3 cache, so that the buildspec.yml can use cache and dynamic environment vars. (#16)
* Added s3 caching option, with autodelete cache * Added s3 caching option, with autodelete cache * Added s3 caching option, with autodelete cache * fixed error on conditional cache usage * Updated the cache_enabled option for clarity Added the ability to dynamically add environment variables. Set the s3 bucket to only create if the cache is enabled Added a null provider verion to remove the warning. Added a random provider with version to remove warning. Added a random string generator to append to bucket names to add uniqueness, lowering the chance of bucket name conflicts * removed build harness file * updated the environment_variables to use a default list containing a map. When it was empty, it caused an error * Fixed formatting for lint * updated variable description * fixed format * fixed source of label module * fixed source of label module * fixed source of label module * set label to use terraform-null-label * Switched to a tagged release of terraform-terraform-label * added git:: to source * moved provider section out of module, and added a section to README to make users aware of the providers used, nd how to squash the warning
1 parent 799e13d commit b0e5d5a

File tree

4 files changed

+193
-49
lines changed

4 files changed

+193
-49
lines changed

‎README.md‎

Lines changed: 54 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
12
# terraform-aws-codebuild [![Build Status](https://travis-ci.org/cloudposse/terraform-aws-codebuild.svg)](https://travis-ci.org/cloudposse/terraform-aws-codebuild)
23

34
Terraform module to create AWS CodeBuild project for AWS CodePipeline
@@ -13,7 +14,7 @@ module "build" {
1314
name = "ci"
1415
stage = "staging"
1516
16-
# http://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref.html
17+
# https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-available.html
1718
build_image = "aws/codebuild/docker:1.12.1"
1819
build_compute_type = "BUILD_GENERAL1_SMALL"
1920
@@ -27,36 +28,68 @@ module "build" {
2728
aws_account_id = "xxxxxxxxxx"
2829
image_repo_name = "ecr-repo-name"
2930
image_tag = "latest"
31+
32+
# Optional extra environment variables
33+
environment_variables = [{
34+
name = "JENKINS_URL"
35+
value = "https://jenkins.example.com"
36+
},
37+
{
38+
name = "COMPANY_NAME"
39+
value = "Amazon"
40+
},
41+
{
42+
name = "TIME_ZONE"
43+
value = "Pacific/Auckland"
44+
45+
}]
3046
}
3147
```
3248

49+
## To hide warnings about unset versions in providers
50+
Add this in your .tf files
3351

34-
## Input
52+
```
53+
provider "random" {
54+
version = "~> 1.0"
55+
}
3556
36-
| Name | Default | Description |
37-
|:--------------------|:----------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------|
38-
| namespace | global | Namespace |
39-
| stage | default | Stage |
40-
| name | codebuild | Name |
41-
| build_image | aws/codebuild/docker:1.12.1 | Docker image for build environment, _e.g._ `aws/codebuild/docker:1.12.1` or `aws/codebuild/eb-nodejs-6.10.0-amazonlinux-64:4.0.0` |
42-
| build_compute_type | BUILD_GENERAL1_SMALL | `CodeBuild` instance size. Possible values are: ```BUILD_GENERAL1_SMALL``` ```BUILD_GENERAL1_MEDIUM``` ```BUILD_GENERAL1_LARGE``` |
43-
| buildspec | "" | (Optional) `buildspec` declaration to use for building the project |
44-
| privileged_mode | "" | (Optional) If set to true, enables running the Docker daemon inside a Docker container on the `CodeBuild` instance. Used when building Docker images |
45-
| aws_region | "" | (Optional) AWS Region, _e.g._ `us-east-1`. Used as `CodeBuild` ENV variable when building Docker images |
46-
| aws_account_id | "" | (Optional) AWS Account ID. Used as `CodeBuild` ENV variable when building Docker images |
47-
| image_repo_name | "UNSET" | (Optional) ECR repository name to store the Docker image built by this module. Used as `CodeBuild` ENV variable when building Docker images |
48-
| image_tag | "latest" | (Optional) Docker image tag in the ECR repository, _e.g._ `latest`. Used as `CodeBuild` ENV variable when building Docker images |
49-
| github_token | "" | (Optional) GitHub auth token environment variable (`GITHUB_TOKEN`) |
57+
provider "null" {
58+
version = "~> 1.0"
59+
}
60+
```
5061

62+
## Input
63+
64+
| Name | Default | Description |
65+
|:----------------------|:----------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------|
66+
| namespace | global | Namespace |
67+
| stage | default | Stage |
68+
| name | codebuild | Name |
69+
| build_image | aws/codebuild/docker:1.12.1 | Docker image for build environment, _e.g._ `aws/codebuild/docker:1.12.1` or `aws/codebuild/eb-nodejs-6.10.0-amazonlinux-64:4.0.0` (use `aws codebuild list-curated-environment-images` to get full list) |
70+
| build_compute_type | BUILD_GENERAL1_SMALL | `CodeBuild` instance size. Possible values are: ```BUILD_GENERAL1_SMALL``` ```BUILD_GENERAL1_MEDIUM``` ```BUILD_GENERAL1_LARGE``` |
71+
| buildspec | "" | (Optional) `buildspec` declaration to use for building the project |
72+
| privileged_mode | "" | (Optional) If set to true, enables running the Docker daemon inside a Docker container on the `CodeBuild` instance. Used when building Docker images |
73+
| aws_region | "" | (Optional) AWS Region, _e.g._ `us-east-1`. Used as `CodeBuild` ENV variable when building Docker images |
74+
| aws_account_id | "" | (Optional) AWS Account ID. Used as `CodeBuild` ENV variable when building Docker images |
75+
| image_repo_name | "UNSET" | (Optional) ECR repository name to store the Docker image built by this module. Used as `CodeBuild` ENV variable when building Docker images |
76+
| image_tag | "latest" | (Optional) Docker image tag in the ECR repository, _e.g._ `latest`. Used as `CodeBuild` ENV variable when building Docker images |
77+
| github_token | "" | (Optional) GitHub auth token environment variable (`GITHUB_TOKEN`) |
78+
| cache_enabled | "true" | (Optional) Creates an S3 bucket, with permissions which allow [CodeBuild to store cache objects](https://aws.amazon.com/blogs/devops/how-to-enable-caching-for-aws-codebuild/) |
79+
| cache_expiration_days | "7" | (Optional) Sets S3 policy to expire objects after X days. |
80+
| cache_bucket_suffix_enabled | "true" | (Optional) Generates an optional 13 character bucket suffix, to help ensure that the bucket will be globally unique |
81+
| environment_variables | [] | (Optional) A list of maps that contain both "name" and "value" keys for adding additional environment variables at build time |
5182

5283

5384
## Output
5485

55-
| Name | Decription |
56-
|:-------------|:-----------------------|
57-
| project_name | CodeBuild project name |
58-
| project_id | CodeBuild project ARN |
59-
| role_arn | IAM Role ARN |
86+
| Name | Decription |
87+
|:----------------------|:--------------------------|
88+
| project_name | CodeBuild project name |
89+
| project_id | CodeBuild project ARN |
90+
| role_arn | IAM Role ARN |
91+
| cache_bucket_name | Name of s3 caching bucket |
92+
6093

6194

6295
## License

‎main.tf‎

Lines changed: 109 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ data "aws_region" "default" {}
44

55
# Define composite variables for resources
66
module "label" {
7-
source = "git::https://github.com/cloudposse/terraform-null-label.git?ref=tags/0.3.1"
7+
source = "git::https://github.com/cloudposse/terraform-terraform-label.git?ref=tags/0.1.0"
88
namespace = "${var.namespace}"
99
name = "${var.name}"
1010
stage = "${var.stage}"
@@ -13,6 +13,57 @@ module "label" {
1313
tags = "${var.tags}"
1414
}
1515

16+
resource "aws_s3_bucket" "cache_bucket" {
17+
count = "${var.cache_enabled == "true" ? 1 : 0}"
18+
bucket = "${local.cache_bucket_name_normalised}"
19+
acl = "private"
20+
force_destroy = true
21+
tags = "${module.label.tags}"
22+
23+
lifecycle_rule {
24+
id = "codebuildcache"
25+
enabled = true
26+
27+
prefix = "/"
28+
tags = "${module.label.tags}"
29+
30+
expiration {
31+
days = "${var.cache_expiration_days}"
32+
}
33+
}
34+
}
35+
36+
resource "random_string" "bucket_prefix" {
37+
length = 12
38+
number = false
39+
upper = false
40+
special = false
41+
lower = true
42+
}
43+
44+
locals {
45+
cache_bucket_name = "${module.label.id}${var.cache_bucket_suffix_enabled == "true" ? "-${random_string.bucket_prefix.result}" : "" }"
46+
47+
## Clean up the bucket name to use only hyphens, and trim its length to 63 characters.
48+
## As per https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html
49+
cache_bucket_name_normalised = "${substr(join("-", split("_", lower(local.cache_bucket_name))), 0, min(length(local.cache_bucket_name),63))}"
50+
51+
## This is the magic where a map of a list of maps is generated
52+
## and used to conditionally add the cache bucket option to the
53+
## aws_codebuild_project
54+
cache_def = {
55+
"true" = [{
56+
type = "S3"
57+
location = "${var.cache_enabled == "true" ? aws_s3_bucket.cache_bucket.0.bucket : "none" }"
58+
}]
59+
60+
"false" = []
61+
}
62+
63+
# Final Map Selected from above
64+
cache = "${local.cache_def[var.cache_enabled]}"
65+
}
66+
1667
resource "aws_iam_role" "default" {
1768
name = "${module.label.id}"
1869
assume_role_policy = "${data.aws_iam_policy_document.role.json}"
@@ -41,6 +92,13 @@ resource "aws_iam_policy" "default" {
4192
policy = "${data.aws_iam_policy_document.permissions.json}"
4293
}
4394

95+
resource "aws_iam_policy" "default_cache_bucket" {
96+
count = "${var.cache_enabled == "true" ? 1 : 0}"
97+
name = "${module.label.id}-cache-bucket"
98+
path = "/service-role/"
99+
policy = "${data.aws_iam_policy_document.permissions_cache_bucket.json}"
100+
}
101+
44102
data "aws_iam_policy_document" "permissions" {
45103
statement {
46104
sid = ""
@@ -65,11 +123,34 @@ data "aws_iam_policy_document" "permissions" {
65123
}
66124
}
67125

126+
data "aws_iam_policy_document" "permissions_cache_bucket" {
127+
statement {
128+
sid = ""
129+
130+
actions = [
131+
"s3:*",
132+
]
133+
134+
effect = "Allow"
135+
136+
resources = [
137+
"${aws_s3_bucket.cache_bucket.arn}",
138+
"${aws_s3_bucket.cache_bucket.arn}/*",
139+
]
140+
}
141+
}
142+
68143
resource "aws_iam_role_policy_attachment" "default" {
69144
policy_arn = "${aws_iam_policy.default.arn}"
70145
role = "${aws_iam_role.default.id}"
71146
}
72147

148+
resource "aws_iam_role_policy_attachment" "default_cache_bucket" {
149+
count = "${var.cache_enabled == "true" ? 1 : 0}"
150+
policy_arn = "${element(aws_iam_policy.default_cache_bucket.*.arn, count.index)}"
151+
role = "${aws_iam_role.default.id}"
152+
}
153+
73154
resource "aws_codebuild_project" "default" {
74155
name = "${module.label.id}"
75156
service_role = "${aws_iam_role.default.arn}"
@@ -78,41 +159,41 @@ resource "aws_codebuild_project" "default" {
78159
type = "CODEPIPELINE"
79160
}
80161

162+
# The cache as a list with a map object inside.
163+
cache = ["${local.cache}"]
164+
81165
environment {
82166
compute_type = "${var.build_compute_type}"
83167
image = "${var.build_image}"
84168
type = "LINUX_CONTAINER"
85169
privileged_mode = "${var.privileged_mode}"
86170

87-
environment_variable {
171+
environment_variable = [{
88172
"name" = "AWS_REGION"
89173
"value" = "${signum(length(var.aws_region)) == 1 ? var.aws_region : data.aws_region.default.name}"
90-
}
91-
92-
environment_variable {
93-
"name" = "AWS_ACCOUNT_ID"
94-
"value" = "${signum(length(var.aws_account_id)) == 1 ? var.aws_account_id : data.aws_caller_identity.default.account_id}"
95-
}
96-
97-
environment_variable {
98-
"name" = "IMAGE_REPO_NAME"
99-
"value" = "${signum(length(var.image_repo_name)) == 1 ? var.image_repo_name : "UNSET"}"
100-
}
101-
102-
environment_variable {
103-
"name" = "IMAGE_TAG"
104-
"value" = "${signum(length(var.image_tag)) == 1 ? var.image_tag : "latest"}"
105-
}
106-
107-
environment_variable {
108-
"name" = "STAGE"
109-
"value" = "${var.stage}"
110-
}
111-
112-
environment_variable {
113-
"name" = "GITHUB_TOKEN"
114-
"value" = "${var.github_token}"
115-
}
174+
},
175+
{
176+
"name" = "AWS_ACCOUNT_ID"
177+
"value" = "${signum(length(var.aws_account_id)) == 1 ? var.aws_account_id : data.aws_caller_identity.default.account_id}"
178+
},
179+
{
180+
"name" = "IMAGE_REPO_NAME"
181+
"value" = "${signum(length(var.image_repo_name)) == 1 ? var.image_repo_name : "UNSET"}"
182+
},
183+
{
184+
"name" = "IMAGE_TAG"
185+
"value" = "${signum(length(var.image_tag)) == 1 ? var.image_tag : "latest"}"
186+
},
187+
{
188+
"name" = "STAGE"
189+
"value" = "${signum(length(var.stage)) == 1 ? var.stage : "UNSET"}"
190+
},
191+
{
192+
"name" = "GITHUB_TOKEN"
193+
"value" = "${signum(length(var.github_token)) == 1 ? var.github_token : "UNSET"}"
194+
},
195+
"${var.environment_variables}",
196+
]
116197
}
117198

118199
source {

‎outputs.tf‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,7 @@ output "project_id" {
99
output "role_arn" {
1010
value = "${aws_iam_role.default.id}"
1111
}
12+
13+
output "cache_bucket_name" {
14+
value = "${var.cache_enabled == "true" ? aws_s3_bucket.cache_bucket.0.bucket : "UNSET" }"
15+
}

‎variables.tf‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,32 @@ variable "name" {
1010
default = "codebuild"
1111
}
1212

13+
variable "environment_variables" {
14+
type = "list"
15+
16+
default = [{
17+
"name" = "NO_ADDITIONAL_BUILD_VARS"
18+
"value" = "TRUE"
19+
}]
20+
21+
description = "A list of maps, that contain both the key 'name' and the key 'value' to be used as additional environment variables for the build."
22+
}
23+
24+
variable "cache_enabled" {
25+
default = "true"
26+
description = "If cache_enabled is true, create an S3 bucket for storing codebuild cache inside"
27+
}
28+
29+
variable "cache_expiration_days" {
30+
default = "7"
31+
description = "How many days should the build cache be kept."
32+
}
33+
34+
variable "cache_bucket_suffix_enabled" {
35+
default = "true"
36+
description = "The cache bucket generates a random 13 character string to generate a unique bucket name. If set to false it uses terraform-null-label's id value"
37+
}
38+
1339
variable "build_image" {
1440
default = "aws/codebuild/docker:1.12.1"
1541
description = "Docker image for build environment, e.g. 'aws/codebuild/docker:1.12.1' or 'aws/codebuild/eb-nodejs-6.10.0-amazonlinux-64:4.0.0'. For more info: http://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref.html"

0 commit comments

Comments
 (0)