Skip to content

Commit a98422b

Browse files
bryantbiggsbryantbiggs
authored
refactor: remove existing vpc endpoint configurations from base module and move into sub-module (#635)
1 parent 43edd44 commit a98422b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

61 files changed

+1120
-6000
lines changed

‎.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
repos:
22
- repo: git://github.com/antonbabenko/pre-commit-terraform
3-
rev: v1.48.0
3+
rev: v1.50.0
44
hooks:
55
- id: terraform_fmt
66
- id: terraform_validate

‎README.md

Lines changed: 29 additions & 712 deletions
Large diffs are not rendered by default.

‎UPGRADE-3.0.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Upgrade from v2.x to v3.x
2+
3+
If you have any questions regarding this upgrade process, please consult the `examples` directory:
4+
5+
- [Complete-VPC](https://github.com/terraform-aws-modules/terraform-aws-vpc/tree/master/examples/complete-vpc)
6+
7+
If you find a bug, please open an issue with supporting configuration to reproduce.
8+
9+
## List of backwards incompatible changes
10+
11+
Previously, VPC endpoints were configured as standalone resources with their own set of variables and attributes. Now, this functionality is provided via a module which loops over a map of maps using `for_each` to generate the desired VPC endpoints. Therefore, to maintain the existing set of functionality while upgrading, you will need to perform the following changes:
12+
13+
1. Move the endpoint resource from the main module to the sub-module. The example state move below is valid for all endpoints you might have configured (reference [`complete-vpc`](https://github.com/terraform-aws-modules/terraform-aws-vpc/tree/master/examples/complete-vpc) example for reference), where `ssmmessages` should be updated for and state move performed for each endpoint configured:
14+
15+
```
16+
terraform state mv 'module.vpc.aws_vpc_endpoint.ssm[0]' 'module.vpc_endpoints.aws_vpc_endpoint.this["ssm"]'
17+
terraform state mv 'module.vpc.aws_vpc_endpoint.ssmmessages[0]' 'module.vpc_endpoints.aws_vpc_endpoint.this["ssmmessages"]'
18+
terraform state mv 'module.vpc.aws_vpc_endpoint.ec2[0]' 'module.vpc_endpoints.aws_vpc_endpoint.this["ec2"]'
19+
...
20+
```
21+
22+
2. Remove the gateway endpoint route table association separate resources. The route table associations are now managed in the VPC endpoint resource itself via the map of maps provided to the VPC endpoint sub-module. Perform the necessary removals for each route table association and for S3 and/or DynamoDB depending on your configuration:
23+
24+
```
25+
terraform state rm 'module.vpc.aws_vpc_endpoint_route_table_association.intra_dynamodb[0]'
26+
terraform state rm 'module.vpc.aws_vpc_endpoint_route_table_association.private_dynamodb[0]'
27+
terraform state rm 'module.vpc.aws_vpc_endpoint_route_table_association.public_dynamodb[0]'
28+
...
29+
```
30+
31+
### Variable and output changes
32+
33+
1. Removed variables:
34+
35+
- `enable_*_endpoint`
36+
- `*_endpoint_type`
37+
- `*_endpoint_security_group_ids`
38+
- `*_endpoint_subnet_ids`
39+
- `*_endpoint_private_dns_enabled`
40+
- `*_endpoint_policy`
41+
42+
2. Renamed variables:
43+
44+
See the [VPC endpoint sub-module](modules/vpc-endpoints) for the more information on the variables to utilize for VPC endpoints
45+
46+
3. Removed outputs:
47+
48+
- `vpc_endpoint_*`
49+
50+
4. Renamed outputs:
51+
52+
VPC endpoint outputs are now provided via the VPC endpoint sub-module and can be accessed via lookups. See [`complete-vpc`](https://github.com/terraform-aws-modules/terraform-aws-vpc/tree/master/examples/complete-vpc) for further examples of how to access VPC endpoint attributes from outputs

‎examples/complete-vpc/README.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,20 +21,22 @@ Note that this example may create resources which can cost money (AWS Elastic IP
2121

2222
| Name | Version |
2323
|------|---------|
24-
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.12.21 |
25-
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.10 |
24+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.12.26 |
25+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.15 |
2626

2727
## Providers
2828

2929
| Name | Version |
3030
|------|---------|
31-
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 3.10 |
31+
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 3.15 |
3232

3333
## Modules
3434

3535
| Name | Source | Version |
3636
|------|--------|---------|
3737
| <a name="module_vpc"></a> [vpc](#module\_vpc) | ../../ | |
38+
| <a name="module_vpc_endpoints"></a> [vpc\_endpoints](#module\_vpc\_endpoints) | ../../modules/vpc-endpoints | |
39+
| <a name="module_vpc_endpoints_nocreate"></a> [vpc\_endpoints\_nocreate](#module\_vpc\_endpoints\_nocreate) | ../../modules/vpc-endpoints | |
3840

3941
## Resources
4042

@@ -43,7 +45,7 @@ Note that this example may create resources which can cost money (AWS Elastic IP
4345
| [aws_iam_policy_document.dynamodb_endpoint_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
4446
| [aws_iam_policy_document.generic_endpoint_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
4547
| [aws_security_group.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/security_group) | data source |
46-
| [aws_vpc_endpoint.dynamodb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc_endpoint) | data source |
48+
| [aws_vpc_endpoint_service.dynamodb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc_endpoint_service) | data source |
4749

4850
## Inputs
4951

‎examples/complete-vpc/main.tf

Lines changed: 127 additions & 91 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,27 @@ provider "aws" {
22
region = "eu-west-1"
33
}
44

5-
data "aws_security_group" "default" {
6-
name = "default"
7-
vpc_id = module.vpc.vpc_id
5+
locals {
6+
name = "complete-example"
7+
region = "eu-west-1"
8+
tags = {
9+
Owner = "user"
10+
Environment = "staging"
11+
Name = "complete"
12+
}
813
}
914

15+
################################################################################
16+
# VPC Module
17+
################################################################################
18+
1019
module "vpc" {
1120
source = "../../"
1221

13-
name = "complete-example"
14-
22+
name = local.name
1523
cidr = "20.10.0.0/16" # 10.0.0.0/8 is reserved for EC2-Classic
1624

17-
azs = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
25+
azs = ["${local.region}a", "${local.region}b", "${local.region}c"]
1826
private_subnets = ["20.10.1.0/24", "20.10.2.0/24", "20.10.3.0/24"]
1927
public_subnets = ["20.10.11.0/24", "20.10.12.0/24", "20.10.13.0/24"]
2028
database_subnets = ["20.10.21.0/24", "20.10.22.0/24", "20.10.23.0/24"]
@@ -53,80 +61,6 @@ module "vpc" {
5361
dhcp_options_domain_name = "service.consul"
5462
dhcp_options_domain_name_servers = ["127.0.0.1", "10.10.0.2"]
5563

56-
# VPC endpoint for S3
57-
# Note - S3 Interface type support is only available on AWS provider 3.10 and later
58-
enable_s3_endpoint = true
59-
s3_endpoint_type = "Interface"
60-
s3_endpoint_private_dns_enabled = false
61-
s3_endpoint_security_group_ids = [data.aws_security_group.default.id]
62-
63-
# VPC endpoint for DynamoDB
64-
enable_dynamodb_endpoint = true
65-
dynamodb_endpoint_policy = data.aws_iam_policy_document.dynamodb_endpoint_policy.json
66-
67-
# VPC endpoint for SSM
68-
enable_ssm_endpoint = true
69-
ssm_endpoint_private_dns_enabled = true
70-
ssm_endpoint_security_group_ids = [data.aws_security_group.default.id]
71-
72-
# VPC endpoint for Lambda
73-
enable_lambda_endpoint = true
74-
lambda_endpoint_private_dns_enabled = true
75-
lambda_endpoint_security_group_ids = [data.aws_security_group.default.id]
76-
77-
# VPC endpoint for SSMMESSAGES
78-
enable_ssmmessages_endpoint = true
79-
ssmmessages_endpoint_private_dns_enabled = true
80-
ssmmessages_endpoint_security_group_ids = [data.aws_security_group.default.id]
81-
82-
# VPC Endpoint for EC2
83-
enable_ec2_endpoint = true
84-
ec2_endpoint_policy = data.aws_iam_policy_document.generic_endpoint_policy.json
85-
ec2_endpoint_private_dns_enabled = true
86-
ec2_endpoint_security_group_ids = [data.aws_security_group.default.id]
87-
88-
# VPC Endpoint for EC2MESSAGES
89-
enable_ec2messages_endpoint = true
90-
ec2messages_endpoint_private_dns_enabled = true
91-
ec2messages_endpoint_security_group_ids = [data.aws_security_group.default.id]
92-
93-
# VPC Endpoint for ECR API
94-
enable_ecr_api_endpoint = true
95-
ecr_api_endpoint_policy = data.aws_iam_policy_document.generic_endpoint_policy.json
96-
ecr_api_endpoint_private_dns_enabled = true
97-
ecr_api_endpoint_security_group_ids = [data.aws_security_group.default.id]
98-
99-
# VPC Endpoint for ECR DKR
100-
enable_ecr_dkr_endpoint = true
101-
ecr_dkr_endpoint_policy = data.aws_iam_policy_document.generic_endpoint_policy.json
102-
ecr_dkr_endpoint_private_dns_enabled = true
103-
ecr_dkr_endpoint_security_group_ids = [data.aws_security_group.default.id]
104-
105-
# VPC endpoint for KMS
106-
enable_kms_endpoint = true
107-
kms_endpoint_private_dns_enabled = true
108-
kms_endpoint_security_group_ids = [data.aws_security_group.default.id]
109-
110-
# VPC endpoint for ECS
111-
enable_ecs_endpoint = true
112-
ecs_endpoint_private_dns_enabled = true
113-
ecs_endpoint_security_group_ids = [data.aws_security_group.default.id]
114-
115-
# VPC endpoint for ECS telemetry
116-
enable_ecs_telemetry_endpoint = true
117-
ecs_telemetry_endpoint_private_dns_enabled = true
118-
ecs_telemetry_endpoint_security_group_ids = [data.aws_security_group.default.id]
119-
120-
# VPC endpoint for CodeDeploy
121-
enable_codedeploy_endpoint = true
122-
codedeploy_endpoint_private_dns_enabled = true
123-
codedeploy_endpoint_security_group_ids = [data.aws_security_group.default.id]
124-
125-
# VPC endpoint for CodeDeploy Commands Secure
126-
enable_codedeploy_commands_secure_endpoint = true
127-
codedeploy_commands_secure_endpoint_private_dns_enabled = true
128-
codedeploy_commands_secure_endpoint_security_group_ids = [data.aws_security_group.default.id]
129-
13064
# Default security group - ingress/egress rules cleared to deny all
13165
manage_default_security_group = true
13266
default_security_group_ingress = []
@@ -138,22 +72,124 @@ module "vpc" {
13872
create_flow_log_cloudwatch_iam_role = true
13973
flow_log_max_aggregation_interval = 60
14074

141-
tags = {
142-
Owner = "user"
143-
Environment = "staging"
144-
Name = "complete"
75+
tags = local.tags
76+
}
77+
78+
################################################################################
79+
# VPC Endpoints Module
80+
################################################################################
81+
82+
module "vpc_endpoints" {
83+
source = "../../modules/vpc-endpoints"
84+
85+
vpc_id = module.vpc.vpc_id
86+
security_group_ids = [data.aws_security_group.default.id]
87+
88+
endpoints = {
89+
s3 = {
90+
service = "s3"
91+
tags = { Name = "s3-vpc-endpoint" }
92+
},
93+
dynamodb = {
94+
service = "dynamodb"
95+
service_type = "Gateway"
96+
route_table_ids = flatten([module.vpc.intra_route_table_ids, module.vpc.private_route_table_ids, module.vpc.public_route_table_ids])
97+
policy = data.aws_iam_policy_document.dynamodb_endpoint_policy.json
98+
tags = { Name = "dynamodb-vpc-endpoint" }
99+
},
100+
ssm = {
101+
service = "ssm"
102+
private_dns_enabled = true
103+
subnet_ids = module.vpc.private_subnets
104+
},
105+
ssmmessages = {
106+
service = "ssmmessages"
107+
private_dns_enabled = true
108+
subnet_ids = module.vpc.private_subnets
109+
},
110+
lambda = {
111+
service = "lambda"
112+
private_dns_enabled = true
113+
subnet_ids = module.vpc.private_subnets
114+
},
115+
ecs = {
116+
service = "ecs"
117+
private_dns_enabled = true
118+
subnet_ids = module.vpc.private_subnets
119+
},
120+
ecs_telemetry = {
121+
service = "ecs-telemetry"
122+
private_dns_enabled = true
123+
subnet_ids = module.vpc.private_subnets
124+
},
125+
ec2 = {
126+
service = "ec2"
127+
private_dns_enabled = true
128+
subnet_ids = module.vpc.private_subnets
129+
},
130+
ec2messages = {
131+
service = "ec2messages"
132+
private_dns_enabled = true
133+
subnet_ids = module.vpc.private_subnets
134+
},
135+
ecr_api = {
136+
service = "ecr.api"
137+
private_dns_enabled = true
138+
subnet_ids = module.vpc.private_subnets
139+
policy = data.aws_iam_policy_document.generic_endpoint_policy.json
140+
},
141+
ecr_dkr = {
142+
service = "ecr.dkr"
143+
private_dns_enabled = true
144+
subnet_ids = module.vpc.private_subnets
145+
policy = data.aws_iam_policy_document.generic_endpoint_policy.json
146+
},
147+
kms = {
148+
service = "kms"
149+
private_dns_enabled = true
150+
subnet_ids = module.vpc.private_subnets
151+
},
152+
codedeploy = {
153+
service = "codedeploy"
154+
private_dns_enabled = true
155+
subnet_ids = module.vpc.private_subnets
156+
},
157+
codedeploy_commands_secure = {
158+
service = "codedeploy-commands-secure"
159+
private_dns_enabled = true
160+
subnet_ids = module.vpc.private_subnets
161+
},
145162
}
146163

147-
vpc_endpoint_tags = {
164+
tags = merge(local.tags, {
148165
Project = "Secret"
149166
Endpoint = "true"
150-
}
167+
})
168+
}
169+
170+
module "vpc_endpoints_nocreate" {
171+
source = "../../modules/vpc-endpoints"
172+
173+
create = false
174+
}
175+
176+
################################################################################
177+
# Supporting Resources
178+
################################################################################
179+
180+
data "aws_security_group" "default" {
181+
name = "default"
182+
vpc_id = module.vpc.vpc_id
151183
}
152184

153185
# Data source used to avoid race condition
154-
data "aws_vpc_endpoint" "dynamodb" {
155-
vpc_id = module.vpc.vpc_id
156-
service_name = "com.amazonaws.eu-west-1.dynamodb"
186+
data "aws_vpc_endpoint_service" "dynamodb" {
187+
service = "dynamodb"
188+
189+
filter {
190+
name = "service-type"
191+
values = ["Gateway"]
192+
}
157193
}
158194

159195
data "aws_iam_policy_document" "dynamodb_endpoint_policy" {
@@ -171,7 +207,7 @@ data "aws_iam_policy_document" "dynamodb_endpoint_policy" {
171207
test = "StringNotEquals"
172208
variable = "aws:sourceVpce"
173209

174-
values = [data.aws_vpc_endpoint.dynamodb.id]
210+
values = [data.aws_vpc_endpoint_service.dynamodb.id]
175211
}
176212
}
177213
}
@@ -191,7 +227,7 @@ data "aws_iam_policy_document" "generic_endpoint_policy" {
191227
test = "StringNotEquals"
192228
variable = "aws:sourceVpce"
193229

194-
values = [data.aws_vpc_endpoint.dynamodb.id]
230+
values = [data.aws_vpc_endpoint_service.dynamodb.id]
195231
}
196232
}
197233
}

0 commit comments

Comments
 (0)