Skip to content

Commit 20d357b

Browse files
bobbyilievbobbyiliev
committed
Simplify disk config for end users
1 parent c5a0113 commit 20d357b

File tree

4 files changed

+114
-49
lines changed

4 files changed

+114
-49
lines changed

‎README.md

Lines changed: 37 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,41 @@ You can also set the `AWS_PROFILE` environment variable to the name of the profi
5555
export AWS_PROFILE=your-profile-name
5656
```
5757

58+
## Disk Support for Materialize
59+
60+
This module supports configuring disk supprot for Materialize using NVMe instance storage and OpenEBS and lgalloc.
61+
62+
When using disk support, you need to use instance types from the `r7gd` or `r6gd` family or other instance types with NVMe instance storage.
63+
64+
### Enabling Disk Support
65+
66+
To enable disk support with default settings:
67+
68+
```hcl
69+
enable_disk_support = true
70+
```
71+
72+
This will:
73+
1. Install OpenEBS via Helm
74+
2. Configure NVMe instance store volumes using the bootstrap script
75+
3. Create appropriate storage classes for Materialize
76+
77+
### Advanced Configuration
78+
79+
In case that you need more control over the disk setup:
80+
81+
```hcl
82+
enable_disk_support = true
83+
84+
disk_support_config = {
85+
openebs_version = "4.2.0"
86+
storage_class_name = "custom-storage-class"
87+
storage_class_parameters = {
88+
volgroup = "custom-volume-group"
89+
}
90+
}
91+
```
92+
5893
## Requirements
5994

6095
| Name | Version |
@@ -113,18 +148,18 @@ export AWS_PROFILE=your-profile-name
113148
| <a name="input_db_instance_class"></a> [db\_instance\_class](#input\_db\_instance\_class) | Instance class for the RDS instance | `string` | `"db.t3.large"` | no |
114149
| <a name="input_db_max_allocated_storage"></a> [db\_max\_allocated\_storage](#input\_db\_max\_allocated\_storage) | Maximum storage for autoscaling (in GB) | `number` | `100` | no |
115150
| <a name="input_db_multi_az"></a> [db\_multi\_az](#input\_db\_multi\_az) | Enable multi-AZ deployment for RDS | `bool` | `false` | no |
151+
| <a name="input_disk_support_config"></a> [disk\_support\_config](#input\_disk\_support\_config) | Advanced configuration for disk support (only used when enable\_disk\_support = true) | <pre>object({<br/> install_openebs = optional(bool, true)<br/> run_disk_setup_script = optional(bool, true)<br/> create_storage_class = optional(bool, true)<br/> openebs_version = optional(string, "4.2.0")<br/> openebs_namespace = optional(string, "openebs")<br/> storage_class_name = optional(string, "openebs-lvm-instance-store-ext4")<br/> storage_class_provisioner = optional(string, "local.csi.openebs.io")<br/> storage_class_parameters = optional(object({<br/> storage = optional(string, "lvm")<br/> fsType = optional(string, "ext4")<br/> volgroup = optional(string, "instance-store-vg")<br/> }), {})<br/> })</pre> | `{}` | no |
116152
| <a name="input_enable_bucket_encryption"></a> [enable\_bucket\_encryption](#input\_enable\_bucket\_encryption) | Enable server-side encryption for the S3 bucket | `bool` | `true` | no |
117153
| <a name="input_enable_bucket_versioning"></a> [enable\_bucket\_versioning](#input\_enable\_bucket\_versioning) | Enable versioning for the S3 bucket | `bool` | `true` | no |
118154
| <a name="input_enable_cluster_creator_admin_permissions"></a> [enable\_cluster\_creator\_admin\_permissions](#input\_enable\_cluster\_creator\_admin\_permissions) | To add the current caller identity as an administrator | `bool` | `true` | no |
119-
| <a name="input_enable_disk_setup"></a> [enable\_disk\_setup](#input\_enable\_disk\_setup) | Whether to enable disk setup | `bool` | `true` | no |
155+
| <a name="input_enable_disk_support"></a> [enable\_disk\_support](#input\_enable\_disk\_support) | Enable disk support for Materialize using OpenEBS and NVMe instance storage. When enabled, this configures OpenEBS, runs the disk setup script for NVMe devices, and creates appropriate storage classes. | `bool` | `true` | no |
120156
| <a name="input_enable_monitoring"></a> [enable\_monitoring](#input\_enable\_monitoring) | Enable CloudWatch monitoring | `bool` | `true` | no |
121157
| <a name="input_environment"></a> [environment](#input\_environment) | Environment name (e.g., prod, staging, dev) | `string` | n/a | yes |
122158
| <a name="input_helm_chart"></a> [helm\_chart](#input\_helm\_chart) | Chart name from repository or local path to chart. For local charts, set the path to the chart directory. | `string` | `"materialize-operator"` | no |
123159
| <a name="input_helm_values"></a> [helm\_values](#input\_helm\_values) | Additional Helm values to merge with defaults | `any` | `{}` | no |
124160
| <a name="input_install_aws_load_balancer_controller"></a> [install\_aws\_load\_balancer\_controller](#input\_install\_aws\_load\_balancer\_controller) | Whether to install the AWS Load Balancer Controller | `bool` | `true` | no |
125161
| <a name="input_install_materialize_operator"></a> [install\_materialize\_operator](#input\_install\_materialize\_operator) | Whether to install the Materialize operator | `bool` | `true` | no |
126162
| <a name="input_install_metrics_server"></a> [install\_metrics\_server](#input\_install\_metrics\_server) | Whether to install the metrics-server for the Materialize Console | `bool` | `true` | no |
127-
| <a name="input_install_openebs"></a> [install\_openebs](#input\_install\_openebs) | Whether to install OpenEBS for lgalloc support | `bool` | `true` | no |
128163
| <a name="input_kubernetes_namespace"></a> [kubernetes\_namespace](#input\_kubernetes\_namespace) | The Kubernetes namespace for the Materialize resources | `string` | `"materialize-environment"` | no |
129164
| <a name="input_log_group_name_prefix"></a> [log\_group\_name\_prefix](#input\_log\_group\_name\_prefix) | Prefix for the CloudWatch log group name (will be combined with environment name) | `string` | `"materialize"` | no |
130165
| <a name="input_materialize_instances"></a> [materialize\_instances](#input\_materialize\_instances) | Configuration for Materialize instances. Due to limitations in Terraform, `materialize_instances` cannot be defined on the first `terraform apply`. | <pre>list(object({<br/> name = string<br/> namespace = optional(string)<br/> database_name = string<br/> environmentd_version = optional(string, "v0.130.4")<br/> cpu_request = optional(string, "1")<br/> memory_request = optional(string, "1Gi")<br/> memory_limit = optional(string, "1Gi")<br/> create_database = optional(bool, true)<br/> create_nlb = optional(bool, true)<br/> internal_nlb = optional(bool, true)<br/> enable_cross_zone_load_balancing = optional(bool, true)<br/> in_place_rollout = optional(bool, false)<br/> request_rollout = optional(string)<br/> force_rollout = optional(string)<br/> balancer_memory_request = optional(string, "256Mi")<br/> balancer_memory_limit = optional(string, "256Mi")<br/> balancer_cpu_request = optional(string, "100m")<br/> }))</pre> | `[]` | no |
@@ -147,10 +182,6 @@ export AWS_PROFILE=your-profile-name
147182
| <a name="input_public_subnet_cidrs"></a> [public\_subnet\_cidrs](#input\_public\_subnet\_cidrs) | CIDR blocks for public subnets | `list(string)` | <pre>[<br/> "10.0.101.0/24",<br/> "10.0.102.0/24",<br/> "10.0.103.0/24"<br/>]</pre> | no |
148183
| <a name="input_service_account_name"></a> [service\_account\_name](#input\_service\_account\_name) | Name of the service account | `string` | `"12345678-1234-1234-1234-123456789012"` | no |
149184
| <a name="input_single_nat_gateway"></a> [single\_nat\_gateway](#input\_single\_nat\_gateway) | Use a single NAT Gateway for all private subnets | `bool` | `false` | no |
150-
| <a name="input_storage_class_create"></a> [storage\_class\_create](#input\_storage\_class\_create) | Whether to create the storage class | `bool` | `true` | no |
151-
| <a name="input_storage_class_name"></a> [storage\_class\_name](#input\_storage\_class\_name) | Name of the storage class | `string` | `"openebs-lvm-instance-store-ext4"` | no |
152-
| <a name="input_storage_class_parameters"></a> [storage\_class\_parameters](#input\_storage\_class\_parameters) | Parameters for the storage class | <pre>object({<br/> storage = string<br/> fsType = string<br/> volgroup = string<br/> })</pre> | <pre>{<br/> "fsType": "ext4",<br/> "storage": "lvm",<br/> "volgroup": "instance-store-vg"<br/>}</pre> | no |
153-
| <a name="input_storage_class_provisioner"></a> [storage\_class\_provisioner](#input\_storage\_class\_provisioner) | Storage class provisioner | `string` | `"local.csi.openebs.io"` | no |
154185
| <a name="input_tags"></a> [tags](#input\_tags) | Default tags to apply to all resources | `map(string)` | <pre>{<br/> "Environment": "dev",<br/> "Project": "materialize",<br/> "Terraform": "true"<br/>}</pre> | no |
155186
| <a name="input_use_local_chart"></a> [use\_local\_chart](#input\_use\_local\_chart) | Whether to use a local chart instead of one from a repository | `bool` | `false` | no |
156187
| <a name="input_vpc_cidr"></a> [vpc\_cidr](#input\_vpc\_cidr) | CIDR block for VPC | `string` | `"10.0.0.0/16"` | no |

‎docs/header.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,3 +53,38 @@ You can also set the `AWS_PROFILE` environment variable to the name of the profi
5353
```bash
5454
export AWS_PROFILE=your-profile-name
5555
```
56+
57+
## Disk Support for Materialize
58+
59+
This module supports configuring disk supprot for Materialize using NVMe instance storage and OpenEBS and lgalloc.
60+
61+
When using disk support, you need to use instance types from the `r7gd` or `r6gd` family or other instance types with NVMe instance storage.
62+
63+
### Enabling Disk Support
64+
65+
To enable disk support with default settings:
66+
67+
```hcl
68+
enable_disk_support = true
69+
```
70+
71+
This will:
72+
1. Install OpenEBS via Helm
73+
2. Configure NVMe instance store volumes using the bootstrap script
74+
3. Create appropriate storage classes for Materialize
75+
76+
### Advanced Configuration
77+
78+
In case that you need more control over the disk setup:
79+
80+
```hcl
81+
enable_disk_support = true
82+
83+
disk_support_config = {
84+
openebs_version = "4.2.0"
85+
storage_class_name = "custom-storage-class"
86+
storage_class_parameters = {
87+
volgroup = "custom-volume-group"
88+
}
89+
}
90+
```

‎main.tf

Lines changed: 25 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,11 @@ module "eks" {
3434
cluster_enabled_log_types = var.cluster_enabled_log_types
3535
node_group_capacity_type = var.node_group_capacity_type
3636
enable_cluster_creator_admin_permissions = var.enable_cluster_creator_admin_permissions
37-
install_openebs = var.install_openebs
38-
enable_disk_setup = var.enable_disk_setup
37+
38+
install_openebs = local.disk_config.install_openebs
39+
enable_disk_setup = local.disk_config.run_disk_setup_script
40+
openebs_namespace = local.disk_config.openebs_namespace
41+
openebs_version = local.disk_config.openebs_version
3942

4043
tags = local.common_tags
4144

@@ -189,10 +192,10 @@ locals {
189192
}
190193
storage = {
191194
storageClass = {
192-
create = var.storage_class_create
193-
name = var.storage_class_name
194-
provisioner = var.storage_class_provisioner
195-
parameters = var.storage_class_parameters
195+
create = local.disk_config.create_storage_class
196+
name = local.disk_config.storage_class_name
197+
provisioner = local.disk_config.storage_class_provisioner
198+
parameters = local.disk_config.storage_class_parameters
196199
}
197200
}
198201
}
@@ -251,6 +254,22 @@ locals {
251254
ManagedBy = "terraform"
252255
}
253256
)
257+
258+
# Disk support configuration
259+
disk_config = {
260+
install_openebs = var.enable_disk_support ? lookup(var.disk_support_config, "install_openebs", true) : false
261+
run_disk_setup_script = var.enable_disk_support ? lookup(var.disk_support_config, "run_disk_setup_script", true) : false
262+
create_storage_class = var.enable_disk_support ? lookup(var.disk_support_config, "create_storage_class", true) : false
263+
openebs_version = lookup(var.disk_support_config, "openebs_version", "4.2.0")
264+
openebs_namespace = lookup(var.disk_support_config, "openebs_namespace", "openebs")
265+
storage_class_name = lookup(var.disk_support_config, "storage_class_name", "openebs-lvm-instance-store-ext4")
266+
storage_class_provisioner = lookup(var.disk_support_config, "storage_class_provisioner", "local.csi.openebs.io")
267+
storage_class_parameters = {
268+
storage = try(var.disk_support_config.storage_class_parameters.storage, "lvm")
269+
fsType = try(var.disk_support_config.storage_class_parameters.fsType, "ext4")
270+
volgroup = try(var.disk_support_config.storage_class_parameters.volgroup, "instance-store-vg")
271+
}
272+
}
254273
}
255274

256275
resource "aws_cloudwatch_log_group" "materialize" {

‎variables.tf

Lines changed: 17 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -364,47 +364,27 @@ variable "install_metrics_server" {
364364
default = true
365365
}
366366

367-
variable "install_openebs" {
368-
description = "Whether to install OpenEBS for lgalloc support"
367+
variable "enable_disk_support" {
368+
description = "Enable disk support for Materialize using OpenEBS and NVMe instance storage. When enabled, this configures OpenEBS, runs the disk setup script for NVMe devices, and creates appropriate storage classes."
369369
type = bool
370370
default = true
371371
}
372372

373-
variable "enable_disk_setup" {
374-
description = "Whether to enable disk setup"
375-
type = bool
376-
default = true
377-
}
378-
379-
variable "storage_class_create" {
380-
description = "Whether to create the storage class"
381-
type = bool
382-
default = true
383-
}
384-
385-
variable "storage_class_name" {
386-
description = "Name of the storage class"
387-
type = string
388-
default = "openebs-lvm-instance-store-ext4"
389-
}
390-
391-
variable "storage_class_provisioner" {
392-
description = "Storage class provisioner"
393-
type = string
394-
default = "local.csi.openebs.io"
395-
}
396-
397-
variable "storage_class_parameters" {
398-
description = "Parameters for the storage class"
373+
variable "disk_support_config" {
374+
description = "Advanced configuration for disk support (only used when enable_disk_support = true)"
399375
type = object({
400-
storage = string
401-
fsType = string
402-
volgroup = string
376+
install_openebs = optional(bool, true)
377+
run_disk_setup_script = optional(bool, true)
378+
create_storage_class = optional(bool, true)
379+
openebs_version = optional(string, "4.2.0")
380+
openebs_namespace = optional(string, "openebs")
381+
storage_class_name = optional(string, "openebs-lvm-instance-store-ext4")
382+
storage_class_provisioner = optional(string, "local.csi.openebs.io")
383+
storage_class_parameters = optional(object({
384+
storage = optional(string, "lvm")
385+
fsType = optional(string, "ext4")
386+
volgroup = optional(string, "instance-store-vg")
387+
}), {})
403388
})
404-
405-
default = {
406-
storage = "lvm"
407-
fsType = "ext4"
408-
volgroup = "instance-store-vg"
409-
}
389+
default = {}
410390
}

0 commit comments

Comments
 (0)