Skip to content

Commit c8a4c97

Browse files
bryantbiggsbryantbiggs
authored
feat!: Refactor module to align with current standards, add missing addtional Redshift resources to module (#61)
1 parent 19a148a commit c8a4c97

File tree

12 files changed

+1896
-395
lines changed

12 files changed

+1896
-395
lines changed

‎.pre-commit-config.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
repos:
22
- repo: https://github.com/antonbabenko/pre-commit-terraform
3-
rev: v1.62.3
3+
rev: v1.72.1
44
hooks:
55
- id: terraform_fmt
66
- id: terraform_validate
@@ -23,7 +23,7 @@ repos:
2323
- '--args=--only=terraform_standard_module_structure'
2424
- '--args=--only=terraform_workspace_remote'
2525
- repo: https://github.com/pre-commit/pre-commit-hooks
26-
rev: v4.1.0
26+
rev: v4.3.0
2727
hooks:
2828
- id: check-merge-conflict
2929
- id: end-of-file-fixer

‎README.md

Lines changed: 280 additions & 78 deletions
Large diffs are not rendered by default.

‎UPGRADE-4.0.md

Lines changed: 242 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,242 @@
1+
# Upgrade from v3.x to v4.x
2+
3+
Please consult the `examples` directory for reference example configurations. If you find a bug, please open an issue with supporting configuration to reproduce.
4+
5+
## List of backwards incompatible changes
6+
7+
- Minimum supported version of Terraform AWS provider updated to v4.17 to support latest resources
8+
- Minimum supported version of Terraform updated to v1.0
9+
- `create` variable added to control whether all resources are created or not. This means that all resources now use the zeroth index `[0]` in the resource name
10+
11+
## Additional changes
12+
13+
### Added
14+
15+
- Support for generating a random password for the `master_password`
16+
- `aws_redshift_snapshot_schedule` resource including the variables and outputs used to support it
17+
- `aws_redshift_snapshot_schedule_association` resource including the variables and outputs used to support it
18+
- `aws_redshift_scheduled_action` resource including support for creating the IAM role and policies plus the associated variables and outputs to support
19+
- `aws_redshift_usage_limit` resource including the variables and outputs used to support it
20+
- `aws_redshift_authentication_profile` resource including the variables and outputs used to support it
21+
- `aws_redshift_cluster_iam_roles` resource including the variables and outputs used to support it
22+
23+
### Modified
24+
25+
- `number_of_nodes` default value of `3` changed to `1`
26+
- `cluster_version` default value of `"1.0"` changed to `null`
27+
- `master_username` default value of `"awsuser"` added
28+
- `encrypted` default value changed to `true`
29+
- By default, a randomly generated password of length `16` is used for the `master_password`
30+
- `master_password` variable marked as `sensitive`
31+
32+
### Removed
33+
34+
- None
35+
36+
### Variable and output changes
37+
38+
1. Removed variables:
39+
40+
- Parameter Group
41+
- `wlm_json_configuration`, `require_ssl`, `use_fips_ssl`, `enable_user_activity_logging`, `max_concurrency_scaling_clusters`, `enable_case_sensitive_identifier` have been replaced by the use of `parameter_group_parameters` where any/all of these values can be set as well as others not listed here.
42+
43+
2. Renamed variables:
44+
45+
- Cluster
46+
- Cluster variables that per the AWS provider do not start with `cluster_` have been renamed to remove the `cluster_` prefix.
47+
- `enable_logging`, `logging_bucket_name`, and `logging_s3_key_prefix` have been replaced with the top level variable `logging` where their equivalent parameters `enable`, `bucket_name`, and `s3_key_prefix` are set, and support for new parameters `log_destination_type`, and `log_exports` have been added.
48+
- `snapshot_copy_destination_region`, `automated_snapshot_retention_period`, `snapshot_copy_grant_name` have been replaced with the top level variable `snapshot_copy` where their equivalent parameters `destination_region`, `retention_period`, and `grant_name` are set.
49+
- `iam_roles` has been renamed to `iam_role_arns` to match API of `aws_redshift_cluster_iam_roles` resource now used
50+
51+
- Parameter Group
52+
- `cluster_parameter_group` ->`parameter_group_family`
53+
54+
- Subnet Group
55+
- `redshift_subnet_group_name` -> `subnet_group_name`: Note: this was not previously used in the manner it was intended. The `cluster_identifier` was used as the name of the subnet group. This has now been corrected
56+
- `subnets` -> `subnet_ids` to match AWS provider
57+
58+
3. Added variables:
59+
60+
## Cluster
61+
- `create` which affects all resources
62+
- `create_random_password` and `random_password_length` to support generating a random password for the `master_password`
63+
- `apply_immediately`
64+
- `aqua_configuration_status`
65+
- `availability_zone`
66+
- `availability_zone_relocation_enabled`
67+
- `default_iam_role_arn`
68+
- `maintenance_track_name`
69+
- `manual_snapshot_retention_period`
70+
- `cluster_timeouts` to support setting `create`, `update`, and `delete` timeout durations
71+
72+
- Parameter Group
73+
- `create_parameter_group` was added to replace `length(var.parameter_group_name) > 0` logic
74+
- `parameter_group_name`
75+
- `parameter_group_parameters` which allows users to set any number of parameters, replacing the previously hardcoed parameters
76+
- `parameter_group_tags`
77+
78+
- Subnet Group
79+
- `create_subnet_group` was added to replace `var.redshift_subnet_group_name == ""` logic
80+
- `subnet_group_description` was added to replace the hardcoded description used previously
81+
- `subnet_group_tags`
82+
83+
4. Removed outputs:
84+
85+
- None
86+
87+
5. Renamed outputs:
88+
89+
- The preceding `redshift_` prefix has been removed from all outputs
90+
91+
6. Added outputs:
92+
93+
- `cluster_dns_name`
94+
- `parameter_group_arn`
95+
- `subnet_group_arn`
96+
97+
## Upgrade Migrations
98+
99+
### Before v3.x Example
100+
101+
```hcl
102+
module "redshift" {
103+
source = "terraform-aws-modules/redshift/aws"
104+
version = "3.4.1"
105+
106+
cluster_identifier = local.name
107+
cluster_node_type = "dc2.large"
108+
cluster_number_of_nodes = 1
109+
110+
cluster_database_name = "mydb"
111+
cluster_master_username = "mydbuser"
112+
cluster_master_password = "MySecretPassw0rd"
113+
114+
subnets = module.vpc.redshift_subnets
115+
vpc_security_group_ids = [module.sg.security_group_id]
116+
}
117+
```
118+
119+
### After v4.x Example
120+
121+
```hcl
122+
module "redshift" {
123+
source = "terraform-aws-modules/redshift/aws"
124+
version = "4.0.0"
125+
126+
cluster_identifier = local.name
127+
node_type = "dc2.large"
128+
number_of_nodes = 1
129+
130+
database_name = "mydb"
131+
master_username = "mydbuser"
132+
master_password = "MySecretPassw0rd"
133+
134+
subnet_ids = module.vpc.redshift_subnets
135+
vpc_security_group_ids = [module.sg.security_group_id]
136+
137+
# Maintain v3.x settings
138+
encrypted = false
139+
automated_snapshot_retention_period = 0
140+
parameter_group_name = "${local.name}-redshift-1-0-custom-params"
141+
parameter_group_parameters = {
142+
wlm_json_configuration = {
143+
name = "wlm_json_configuration"
144+
value = jsonencode([
145+
{
146+
query_concurrency = 5
147+
}
148+
])
149+
}
150+
require_ssl = {
151+
name = "require_ssl"
152+
value = false
153+
}
154+
use_fips_ssl = {
155+
name = "use_fips_ssl"
156+
value = false
157+
}
158+
enable_user_activity_logging = {
159+
name = "enable_user_activity_logging"
160+
value = false
161+
}
162+
max_concurrency_scaling_clusters = {
163+
name = "max_concurrency_scaling_clusters"
164+
value = 1
165+
}
166+
enable_case_sensitive_identifier = {
167+
name = "enable_case_sensitive_identifier"
168+
value = false
169+
}
170+
}
171+
subnet_group_description = "Redshift subnet group of ${local.name}"
172+
create_random_password = false
173+
}
174+
```
175+
176+
### Diff of Before vs After
177+
178+
```diff
179+
module "redshift" {
180+
source = "terraform-aws-modules/redshift/aws"
181+
- version = "3.4.1"
182+
+ version = "4.0.0"
183+
184+
cluster_identifier = local.name
185+
- cluster_node_type = "dc2.large"
186+
+ node_type = "dc2.large"
187+
- cluster_number_of_nodes = 1
188+
+ number_of_nodes = 1
189+
190+
- cluster_database_name = "mydb"
191+
+ database_name = "mydb"
192+
- cluster_master_username = "mydbuser"
193+
+ master_username = "mydbuser"
194+
- cluster_master_password = "MySecretPassw0rd"
195+
+ master_password = "MySecretPassw0rd"
196+
197+
- subnets = module.vpc.redshift_subnets
198+
+ subnet_ids = module.vpc.redshift_subnets
199+
vpc_security_group_ids = [module.sg.security_group_id]
200+
201+
+ # Maintain v3.x settings
202+
+ encrypted = false
203+
+ automated_snapshot_retention_period = 0
204+
+ parameter_group_name = "${local.name}-redshift-1-0-custom-params"
205+
+ parameter_group_parameters = {
206+
+ wlm_json_configuration = {
207+
+ name = "wlm_json_configuration"
208+
+ value = jsonencode([
209+
+ {
210+
+ query_concurrency = 5
211+
+ }
212+
+ ])
213+
+ }
214+
+ require_ssl = {
215+
+ name = "require_ssl"
216+
+ value = false
217+
+ }
218+
+ use_fips_ssl = {
219+
+ name = "use_fips_ssl"
220+
+ value = false
221+
+ }
222+
+ enable_user_activity_logging = {
223+
+ name = "enable_user_activity_logging"
224+
+ value = false
225+
+ }
226+
+ max_concurrency_scaling_clusters = {
227+
+ name = "max_concurrency_scaling_clusters"
228+
+ value = 1
229+
+ }
230+
+ enable_case_sensitive_identifier = {
231+
+ name = "enable_case_sensitive_identifier"
232+
+ value = false
233+
+ }
234+
+ }
235+
+ subnet_group_description = "Redshift subnet group of ${local.name}"
236+
+ create_random_password = false
237+
}
238+
```
239+
240+
### State Move Commands
241+
242+
None required

‎examples/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Examples
2+
3+
Please note - the examples provided serve two primary means:
4+
5+
1. Show users working examples of the various ways in which the module can be configured and features supported
6+
2. A means of testing/validating module changes
7+
8+
Please do not mistake the examples provided as "best practices". It is up to users to consult the AWS service documentation for best practices, usage recommendations, etc.

‎examples/complete/README.md

Lines changed: 63 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
# Complete Redshift example
22

3-
Configuration in this directory creates VPC with Redshift subnet, security group and Redshift cluster itself.
3+
Configuration in this directory creates AWS Redshift clusters with demonstrating the various methods of configuring/customizing:
4+
5+
- A disabled cluster
6+
- A default, "out of the box" Redshift cluster
7+
- A "complete" cluster demonstrating the broad array of configurations including the use of snapshot copy grants. NOTE: the term "complete" here is used to denote the "compete array of configurations offered" and is not representative of recommended/best practices for provisioning a Redshift cluster. Users should refer to the AWS documenation for recommended practices for provisioning a Redshift cluster.
48

59
## Usage
610

@@ -19,24 +23,40 @@ Note that this example may create resources which cost money. Run `terraform des
1923

2024
| Name | Version |
2125
|------|---------|
22-
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.12.31 |
23-
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 2.25 |
26+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
27+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.17 |
28+
| <a name="requirement_random"></a> [random](#requirement\_random) | >= 3.0 |
2429

2530
## Providers
2631

27-
No providers.
32+
| Name | Version |
33+
|------|---------|
34+
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.17 |
35+
| <a name="provider_aws.us_east_1"></a> [aws.us\_east\_1](#provider\_aws.us\_east\_1) | >= 4.17 |
36+
| <a name="provider_random"></a> [random](#provider\_random) | >= 3.0 |
2837

2938
## Modules
3039

3140
| Name | Source | Version |
3241
|------|--------|---------|
42+
| <a name="module_default"></a> [default](#module\_default) | ../../ | n/a |
43+
| <a name="module_disabled"></a> [disabled](#module\_disabled) | ../../ | n/a |
3344
| <a name="module_redshift"></a> [redshift](#module\_redshift) | ../../ | n/a |
34-
| <a name="module_sg"></a> [sg](#module\_sg) | terraform-aws-modules/security-group/aws//modules/redshift | ~> 4.0 |
45+
| <a name="module_s3_logs"></a> [s3\_logs](#module\_s3\_logs) | terraform-aws-modules/s3-bucket/aws | ~> 3.0 |
46+
| <a name="module_security_group"></a> [security\_group](#module\_security\_group) | terraform-aws-modules/security-group/aws//modules/redshift | ~> 4.0 |
3547
| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 3.0 |
3648

3749
## Resources
3850

39-
No resources.
51+
| Name | Type |
52+
|------|------|
53+
| [aws_kms_key.redshift](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource |
54+
| [aws_kms_key.redshift_us_east_1](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource |
55+
| [aws_redshift_snapshot_copy_grant.useast1](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/redshift_snapshot_copy_grant) | resource |
56+
| [aws_redshift_subnet_group.endpoint](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/redshift_subnet_group) | resource |
57+
| [random_pet.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) | resource |
58+
| [aws_iam_policy_document.s3_redshift](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
59+
| [aws_redshift_service_account.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/redshift_service_account) | data source |
4060

4161
## Inputs
4262

@@ -46,10 +66,41 @@ No inputs.
4666

4767
| Name | Description |
4868
|------|-------------|
49-
| <a name="output_redshift_cluster_endpoint"></a> [redshift\_cluster\_endpoint](#output\_redshift\_cluster\_endpoint) | Redshift endpoint |
50-
| <a name="output_redshift_cluster_hostname"></a> [redshift\_cluster\_hostname](#output\_redshift\_cluster\_hostname) | Redshift hostname |
51-
| <a name="output_redshift_cluster_id"></a> [redshift\_cluster\_id](#output\_redshift\_cluster\_id) | The availability zone of the RDS instance |
52-
| <a name="output_redshift_cluster_port"></a> [redshift\_cluster\_port](#output\_redshift\_cluster\_port) | Redshift port |
53-
| <a name="output_security_group_id"></a> [security\_group\_id](#output\_security\_group\_id) | The ID of the security group |
54-
| <a name="output_vpc_id"></a> [vpc\_id](#output\_vpc\_id) | The ID of the VPC |
69+
| <a name="output_authentication_profiles"></a> [authentication\_profiles](#output\_authentication\_profiles) | Map of authentication profiles created and their associated attributes |
70+
| <a name="output_cluster_arn"></a> [cluster\_arn](#output\_cluster\_arn) | The Redshift cluster ARN |
71+
| <a name="output_cluster_automated_snapshot_retention_period"></a> [cluster\_automated\_snapshot\_retention\_period](#output\_cluster\_automated\_snapshot\_retention\_period) | The backup retention period |
72+
| <a name="output_cluster_availability_zone"></a> [cluster\_availability\_zone](#output\_cluster\_availability\_zone) | The availability zone of the Cluster |
73+
| <a name="output_cluster_database_name"></a> [cluster\_database\_name](#output\_cluster\_database\_name) | The name of the default database in the Cluster |
74+
| <a name="output_cluster_dns_name"></a> [cluster\_dns\_name](#output\_cluster\_dns\_name) | The DNS name of the cluster |
75+
| <a name="output_cluster_encrypted"></a> [cluster\_encrypted](#output\_cluster\_encrypted) | Whether the data in the cluster is encrypted |
76+
| <a name="output_cluster_endpoint"></a> [cluster\_endpoint](#output\_cluster\_endpoint) | The connection endpoint |
77+
| <a name="output_cluster_hostname"></a> [cluster\_hostname](#output\_cluster\_hostname) | The hostname of the Redshift cluster |
78+
| <a name="output_cluster_id"></a> [cluster\_id](#output\_cluster\_id) | The Redshift cluster ID |
79+
| <a name="output_cluster_identifier"></a> [cluster\_identifier](#output\_cluster\_identifier) | The Redshift cluster identifier |
80+
| <a name="output_cluster_node_type"></a> [cluster\_node\_type](#output\_cluster\_node\_type) | The type of nodes in the cluster |
81+
| <a name="output_cluster_nodes"></a> [cluster\_nodes](#output\_cluster\_nodes) | The nodes in the cluster. Each node is a map of the following attributes: `node_role`, `private_ip_address`, and `public_ip_address` |
82+
| <a name="output_cluster_parameter_group_name"></a> [cluster\_parameter\_group\_name](#output\_cluster\_parameter\_group\_name) | The name of the parameter group to be associated with this cluster |
83+
| <a name="output_cluster_port"></a> [cluster\_port](#output\_cluster\_port) | The port the cluster responds on |
84+
| <a name="output_cluster_preferred_maintenance_window"></a> [cluster\_preferred\_maintenance\_window](#output\_cluster\_preferred\_maintenance\_window) | The backup window |
85+
| <a name="output_cluster_public_key"></a> [cluster\_public\_key](#output\_cluster\_public\_key) | The public key for the cluster |
86+
| <a name="output_cluster_revision_number"></a> [cluster\_revision\_number](#output\_cluster\_revision\_number) | The specific revision number of the database in the cluster |
87+
| <a name="output_cluster_security_groups"></a> [cluster\_security\_groups](#output\_cluster\_security\_groups) | The security groups associated with the cluster |
88+
| <a name="output_cluster_subnet_group_name"></a> [cluster\_subnet\_group\_name](#output\_cluster\_subnet\_group\_name) | The name of a cluster subnet group to be associated with this cluster |
89+
| <a name="output_cluster_type"></a> [cluster\_type](#output\_cluster\_type) | The Redshift cluster type |
90+
| <a name="output_cluster_version"></a> [cluster\_version](#output\_cluster\_version) | The version of Redshift engine software |
91+
| <a name="output_cluster_vpc_security_group_ids"></a> [cluster\_vpc\_security\_group\_ids](#output\_cluster\_vpc\_security\_group\_ids) | The VPC security group ids associated with the cluster |
92+
| <a name="output_endpoint_access_address"></a> [endpoint\_access\_address](#output\_endpoint\_access\_address) | The DNS address of the endpoint |
93+
| <a name="output_endpoint_access_id"></a> [endpoint\_access\_id](#output\_endpoint\_access\_id) | The Redshift-managed VPC endpoint name |
94+
| <a name="output_endpoint_access_port"></a> [endpoint\_access\_port](#output\_endpoint\_access\_port) | The port number on which the cluster accepts incoming connections |
95+
| <a name="output_endpoint_access_vpc_endpoint"></a> [endpoint\_access\_vpc\_endpoint](#output\_endpoint\_access\_vpc\_endpoint) | The connection endpoint for connecting to an Amazon Redshift cluster through the proxy. See details below |
96+
| <a name="output_parameter_group_arn"></a> [parameter\_group\_arn](#output\_parameter\_group\_arn) | Amazon Resource Name (ARN) of the parameter group created |
97+
| <a name="output_parameter_group_id"></a> [parameter\_group\_id](#output\_parameter\_group\_id) | The name of the Redshift parameter group created |
98+
| <a name="output_scheduled_action_iam_role_arn"></a> [scheduled\_action\_iam\_role\_arn](#output\_scheduled\_action\_iam\_role\_arn) | Scheduled actions IAM role ARN |
99+
| <a name="output_scheduled_action_iam_role_name"></a> [scheduled\_action\_iam\_role\_name](#output\_scheduled\_action\_iam\_role\_name) | Scheduled actions IAM role name |
100+
| <a name="output_scheduled_action_iam_role_unique_id"></a> [scheduled\_action\_iam\_role\_unique\_id](#output\_scheduled\_action\_iam\_role\_unique\_id) | Stable and unique string identifying the scheduled action IAM role |
101+
| <a name="output_scheduled_actions"></a> [scheduled\_actions](#output\_scheduled\_actions) | A map of maps containing scheduled action details |
102+
| <a name="output_snapshot_schedule_arn"></a> [snapshot\_schedule\_arn](#output\_snapshot\_schedule\_arn) | Amazon Resource Name (ARN) of the Redshift Snapshot Schedule |
103+
| <a name="output_subnet_group_arn"></a> [subnet\_group\_arn](#output\_subnet\_group\_arn) | Amazon Resource Name (ARN) of the Redshift subnet group created |
104+
| <a name="output_subnet_group_id"></a> [subnet\_group\_id](#output\_subnet\_group\_id) | The ID of Redshift Subnet group created |
105+
| <a name="output_usage_limits"></a> [usage\_limits](#output\_usage\_limits) | Map of usage limits created and their associated attributes |
55106
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

0 commit comments

Comments
 (0)