feat: secrets manager integration for ingress

Author: Vipin Kumar

README.md

@@ -255,6 +255,7 @@ Optionally, you need the following permissions to attach Access Management tags
 | Name | Type |
 |------|------|
 | [ibm_container_addons.addons](https://registry.terraform.io/providers/ibm-cloud/ibm/latest/docs/resources/container_addons) | resource |
+| [ibm_container_ingress_instance.instance](https://registry.terraform.io/providers/ibm-cloud/ibm/latest/docs/resources/container_ingress_instance) | resource |
 | [ibm_container_vpc_cluster.autoscaling_cluster](https://registry.terraform.io/providers/ibm-cloud/ibm/latest/docs/resources/container_vpc_cluster) | resource |
 | [ibm_container_vpc_cluster.cluster](https://registry.terraform.io/providers/ibm-cloud/ibm/latest/docs/resources/container_vpc_cluster) | resource |
 | [ibm_container_vpc_worker_pool.autoscaling_pool](https://registry.terraform.io/providers/ibm-cloud/ibm/latest/docs/resources/container_vpc_worker_pool) | resource |
@@ -299,9 +300,12 @@ Optionally, you need the following permissions to attach Access Management tags
 | <a name="input_disable_public_endpoint"></a> [disable\_public\_endpoint](#input\_disable\_public\_endpoint) | Whether access to the public service endpoint is disabled when the cluster is created. Does not affect existing clusters. You can't disable a public endpoint on an existing cluster, so you can't convert a public cluster to a private cluster. To change a public endpoint to private, create another cluster with this input set to `true`. | `bool` | `false` | no |
 | <a name="input_enable_ocp_console"></a> [enable\_ocp\_console](#input\_enable\_ocp\_console) | Flag to specify whether to enable or disable the OpenShift console. If set to `null` the module will not modify the setting currently set on the cluster. Bare in mind when setting this to `true` or `false` on a cluster with private only endpoint enabled, the runtime must be able to access the private endpoint. | `bool` | `null` | no |
 | <a name="input_enable_registry_storage"></a> [enable\_registry\_storage](#input\_enable\_registry\_storage) | Set to `true` to enable IBM Cloud Object Storage for the Red Hat OpenShift internal image registry. Set to `false` only for new cluster deployments in an account that is allowlisted for this feature. | `bool` | `true` | no |
+| <a name="input_enable_secrets_manager_for_ingress"></a> [enable\_secrets\_manager\_for\_ingress](#input\_enable\_secrets\_manager\_for\_ingress) | Whether to enable secrets manager for storing ingress certificate. | `bool` | `false` | no |
 | <a name="input_existing_cos_id"></a> [existing\_cos\_id](#input\_existing\_cos\_id) | The COS id of an already existing COS instance to use for OpenShift internal registry storage. Only required if 'enable\_registry\_storage' and 'use\_existing\_cos' are true. | `string` | `null` | no |
+| <a name="input_existing_secrets_manager_instance_crn"></a> [existing\_secrets\_manager\_instance\_crn](#input\_existing\_secrets\_manager\_instance\_crn) | CRN of secrets manager instance where ingress certificate secrets will be stored. | `string` | `null` | no |
 | <a name="input_force_delete_storage"></a> [force\_delete\_storage](#input\_force\_delete\_storage) | Flag indicating whether or not to delete attached storage when destroying the cluster - Default: false | `bool` | `false` | no |
 | <a name="input_ignore_worker_pool_size_changes"></a> [ignore\_worker\_pool\_size\_changes](#input\_ignore\_worker\_pool\_size\_changes) | Enable if using worker autoscaling. Stops Terraform managing worker count | `bool` | `false` | no |
+| <a name="input_is_default_secrets_manager_instance"></a> [is\_default\_secrets\_manager\_instance](#input\_is\_default\_secrets\_manager\_instance) | Whether the secrets manager instance provided will be default for storing ingress certificates. | `bool` | `true` | no |
 | <a name="input_kms_config"></a> [kms\_config](#input\_kms\_config) | Use to attach a KMS instance to the cluster. If account\_id is not provided, defaults to the account in use. | <pre>object({<br/>    crk_id           = string<br/>    instance_id      = string<br/>    private_endpoint = optional(bool, true) # defaults to true<br/>    account_id       = optional(string)     # To attach KMS instance from another account<br/>    wait_for_apply   = optional(bool, true) # defaults to true so terraform will wait until the KMS is applied to the master, ready and deployed<br/>  })</pre> | `null` | no |
 | <a name="input_manage_all_addons"></a> [manage\_all\_addons](#input\_manage\_all\_addons) | Instructs Terraform to manage all cluster addons, even if addons were installed outside of the module. If set to 'true' this module will destroy any addons that were installed by other sources. | `bool` | `false` | no |
 | <a name="input_number_of_lbs"></a> [number\_of\_lbs](#input\_number\_of\_lbs) | The number of LBs to associated the `additional_lb_security_group_names` security group with. | `number` | `1` | no |
@@ -310,6 +314,7 @@ Optionally, you need the following permissions to attach Access Management tags
 | <a name="input_pod_subnet_cidr"></a> [pod\_subnet\_cidr](#input\_pod\_subnet\_cidr) | Specify a custom subnet CIDR to provide private IP addresses for pods. The subnet must have a CIDR of at least `/23` or larger. Default value is `172.30.0.0/16` when the variable is set to `null`. | `string` | `null` | no |
 | <a name="input_region"></a> [region](#input\_region) | The IBM Cloud region where the cluster will be provisioned. | `string` | n/a | yes |
 | <a name="input_resource_group_id"></a> [resource\_group\_id](#input\_resource\_group\_id) | The Id of an existing IBM Cloud resource group where the cluster will be grouped. | `string` | n/a | yes |
+| <a name="input_secrets_manager_secret_group_id"></a> [secrets\_manager\_secret\_group\_id](#input\_secrets\_manager\_secret\_group\_id) | Secret group id where ingress secrets will be kept in the secrets manager instance. If not specified, default group will be used. | `string` | `""` | no |
 | <a name="input_service_subnet_cidr"></a> [service\_subnet\_cidr](#input\_service\_subnet\_cidr) | Specify a custom subnet CIDR to provide private IP addresses for services. The subnet must be at least `/24` or larger. Default value is `172.21.0.0/16` when the variable is set to `null`. | `string` | `null` | no |
 | <a name="input_tags"></a> [tags](#input\_tags) | Metadata labels describing this cluster deployment, i.e. test | `list(string)` | `[]` | no |
 | <a name="input_use_existing_cos"></a> [use\_existing\_cos](#input\_use\_existing\_cos) | Flag indicating whether or not to use an existing COS instance for OpenShift internal registry storage. Only applicable if 'enable\_registry\_storage' is true | `bool` | `false` | no |
@@ -330,6 +335,7 @@ Optionally, you need the following permissions to attach Access Management tags
 | <a name="output_cluster_name"></a> [cluster\_name](#output\_cluster\_name) | Name of the created cluster |
 | <a name="output_cos_crn"></a> [cos\_crn](#output\_cos\_crn) | CRN of the COS instance |
 | <a name="output_ingress_hostname"></a> [ingress\_hostname](#output\_ingress\_hostname) | The hostname that was assigned to your Ingress subdomain. |
+| <a name="output_ingress_secrets_group"></a> [ingress\_secrets\_group](#output\_ingress\_secrets\_group) | Info of secrets group for storing ingress certificates |
 | <a name="output_kms_config"></a> [kms\_config](#output\_kms\_config) | KMS configuration details |
 | <a name="output_master_status"></a> [master\_status](#output\_master\_status) | The status of the Kubernetes master. |
 | <a name="output_master_url"></a> [master\_url](#output\_master\_url) | The URL of the Kubernetes master. |

ibm_catalog.json

@@ -382,6 +382,18 @@
             },
             {
               "key": "cbr_rules"
+            },
+            {
+              "key": "enable_secrets_manager_for_ingress"
+            },
+            {
+              "key": "existing_secrets_manager_instance_crn"
+            },
+            {
+              "key": "secrets_manager_secret_group_id"
+            },
+            {
+              "key": "is_default_secrets_manager_instance"
             }
           ],
           "dependencies": [

main.tf

@@ -743,3 +743,16 @@ module "cbr_rule" {
   }]
   operations = var.cbr_rules[count.index].operations == null ? local.default_operations : var.cbr_rules[count.index].operations
 }
+
+##############################################################
+# Ingress Secrets Manager Integration
+##############################################################
+
+resource "ibm_container_ingress_instance" "instance" {
+  count           = var.enable_secrets_manager_for_ingress ? 1 : 0
+  depends_on      = [ibm_container_vpc_cluster.cluster, ibm_container_vpc_cluster.autoscaling_cluster, ibm_container_vpc_worker_pool.pool, ibm_container_vpc_worker_pool.autoscaling_pool]
+  cluster         = var.cluster_name
+  instance_crn    = var.existing_secrets_manager_instance_crn
+  is_default      = var.is_default_secrets_manager_instance
+  secret_group_id = var.secrets_manager_secret_group_id
+}

outputs.tf

@@ -104,3 +104,8 @@ output "registry_vpe" {
   description = "Info about the registry VPE, if it exists. For more info about schema, see https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/data-sources/is_virtual_endpoint_gateway"
   value       = local.registry_vpe_id
 }
+
+output "ingress_secrets_group" {
+  description = "Info of secrets group for storing ingress certificates"
+  value       = var.enable_secrets_manager_for_ingress ? ibm_container_ingress_instance.instance[0] : null
+}

solutions/fully-configurable/README.md

@@ -71,16 +71,19 @@ The following resources are provisioned by this example:
 | <a name="input_disable_public_endpoint"></a> [disable\_public\_endpoint](#input\_disable\_public\_endpoint) | Whether access to the public service endpoint is disabled when the cluster is created. Does not affect existing clusters. You can't disable a public endpoint on an existing cluster, so you can't convert a public cluster to a private cluster. To change a public endpoint to private, create another cluster with this input set to `true`. | `bool` | `true` | no |
 | <a name="input_enable_autoscaling_for_default_pool"></a> [enable\_autoscaling\_for\_default\_pool](#input\_enable\_autoscaling\_for\_default\_pool) | Set `true` to enable automatic scaling of worker based on workload demand. | `bool` | `false` | no |
 | <a name="input_enable_ocp_console"></a> [enable\_ocp\_console](#input\_enable\_ocp\_console) | Flag to specify whether to enable or disable the OpenShift console. If set to `null` the module will not modify the setting currently set on the cluster. Bare in mind when setting this to `true` or `false` on a cluster with private only endpoint enabled, the runtime must be able to access the private endpoint. | `bool` | `null` | no |
+| <a name="input_enable_secrets_manager_for_ingress"></a> [enable\_secrets\_manager\_for\_ingress](#input\_enable\_secrets\_manager\_for\_ingress) | Whether to enable secrets manager for storing ingress certificate. | `bool` | `false` | no |
 | <a name="input_existing_boot_volume_kms_key_crn"></a> [existing\_boot\_volume\_kms\_key\_crn](#input\_existing\_boot\_volume\_kms\_key\_crn) | The CRN of an existing KMS key to use to encrypt the the block storage volumes for VPC. If no value is set for this variable, specify a value for either the `existing_kms_instance_crn` variable to create a key ring and key. | `string` | `null` | no |
 | <a name="input_existing_cluster_kms_key_crn"></a> [existing\_cluster\_kms\_key\_crn](#input\_existing\_cluster\_kms\_key\_crn) | The CRN of an existing KMS key to use for encrypting the Object Storage of the Cluster. If no value is set for this variable, please specify a value for `existing_kms_instance_crn` variable to create a key ring and key. | `string` | `null` | no |
 | <a name="input_existing_cos_instance_crn"></a> [existing\_cos\_instance\_crn](#input\_existing\_cos\_instance\_crn) | The CRN of an already existing Object Storage instance to use for OpenShift internal registry storage. | `string` | n/a | yes |
 | <a name="input_existing_kms_instance_crn"></a> [existing\_kms\_instance\_crn](#input\_existing\_kms\_instance\_crn) | The CRN of an existing KMS instance (Hyper Protect Crypto Services or Key Protect). If the KMS instance is in different account you must also provide a value for `ibmcloud_kms_api_key`. | `string` | `null` | no |
 | <a name="input_existing_resource_group_name"></a> [existing\_resource\_group\_name](#input\_existing\_resource\_group\_name) | The name of an existing resource group to provision the cluster. | `string` | `"Default"` | no |
+| <a name="input_existing_secrets_manager_instance_crn"></a> [existing\_secrets\_manager\_instance\_crn](#input\_existing\_secrets\_manager\_instance\_crn) | CRN of secrets manager instance where ingress certificate secrets will be stored. | `string` | `null` | no |
 | <a name="input_existing_subnet_ids"></a> [existing\_subnet\_ids](#input\_existing\_subnet\_ids) | The list of IDs of existing subnets where the default worker pool nodes of the cluster will be provisioned. | `list(string)` | `[]` | no |
 | <a name="input_existing_vpc_crn"></a> [existing\_vpc\_crn](#input\_existing\_vpc\_crn) | The CRN of an existing VPC. If the user provides only the `existing_vpc_crn` the default worker pool will be provisioned across all the subnets in the VPC. | `string` | n/a | yes |
 | <a name="input_ibmcloud_api_key"></a> [ibmcloud\_api\_key](#input\_ibmcloud\_api\_key) | The IBM Cloud api key. | `string` | n/a | yes |
 | <a name="input_ibmcloud_kms_api_key"></a> [ibmcloud\_kms\_api\_key](#input\_ibmcloud\_kms\_api\_key) | The IBM Cloud API key that can create a root key and key ring in the key management service (KMS) instance for the cluster. If not specified, the 'ibmcloud\_api\_key' variable is used. Specify this key if the KMS instance in `existing_kms_instance_crn` is in an account that is different from the cluster's account. Leave this input empty if both the cluster and the KMS instance are in the same account. | `string` | `null` | no |
 | <a name="input_ignore_worker_pool_size_changes"></a> [ignore\_worker\_pool\_size\_changes](#input\_ignore\_worker\_pool\_size\_changes) | Enable if using worker autoscaling. Stops Terraform managing worker count. | `bool` | `false` | no |
+| <a name="input_is_default_secrets_manager_instance"></a> [is\_default\_secrets\_manager\_instance](#input\_is\_default\_secrets\_manager\_instance) | Whether the secrets manager instance provided will be default for storing ingress certificates. | `bool` | `true` | no |
 | <a name="input_kms_encryption_enabled_boot_volume"></a> [kms\_encryption\_enabled\_boot\_volume](#input\_kms\_encryption\_enabled\_boot\_volume) | Set this to true to control the encryption keys used to encrypt the data that for the block storage volumes for VPC. If set to false, the data is encrypted by using randomly generated keys. For more info on encrypting block storage volumes, see https://cloud.ibm.com/docs/vpc?topic=vpc-creating-instances-byok | `bool` | `false` | no |
 | <a name="input_kms_encryption_enabled_cluster"></a> [kms\_encryption\_enabled\_cluster](#input\_kms\_encryption\_enabled\_cluster) | Set to true to enable KMS encryption for the cluster's Object Storage bucket. When set to true, a value must be passed for either `existing_cluster_kms_key_crn` or `existing_kms_instance_crn`. | `bool` | `false` | no |
 | <a name="input_kms_endpoint_type"></a> [kms\_endpoint\_type](#input\_kms\_endpoint\_type) | The endpoint for communicating with the KMS instance. Possible values: `public`, `private`. Applies only if `kms_encryption_enabled_cluster` is true | `string` | `"private"` | no |
@@ -91,6 +94,7 @@ The following resources are provisioned by this example:
 | <a name="input_pod_subnet_cidr"></a> [pod\_subnet\_cidr](#input\_pod\_subnet\_cidr) | Specify a custom subnet CIDR to provide private IP addresses for pods. The subnet must have a CIDR of at least `/23` or larger. Default value is `172.30.0.0/16` when the variable is set to `null`. | `string` | `null` | no |
 | <a name="input_prefix"></a> [prefix](#input\_prefix) | The prefix to add to all resources that this solution creates (e.g `prod`, `test`, `dev`). To not use any prefix value, you can set this value to `null` or an empty string. | `string` | n/a | yes |
 | <a name="input_provider_visibility"></a> [provider\_visibility](#input\_provider\_visibility) | Set the visibility value for the IBM terraform provider. Supported values are `public`, `private`, `public-and-private`. [Learn more](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/guides/custom-service-endpoints). | `string` | `"private"` | no |
+| <a name="input_secrets_manager_secret_group_id"></a> [secrets\_manager\_secret\_group\_id](#input\_secrets\_manager\_secret\_group\_id) | Secret group id where ingress secrets will be kept in the secrets manager instance. If not specified, default group will be used. | `string` | `""` | no |
 | <a name="input_service_subnet_cidr"></a> [service\_subnet\_cidr](#input\_service\_subnet\_cidr) | Specify a custom subnet CIDR to provide private IP addresses for services. The subnet must be at least `/24` or larger. Default value is `172.21.0.0/16` when the variable is set to `null`. | `string` | `null` | no |
 | <a name="input_use_private_endpoint"></a> [use\_private\_endpoint](#input\_use\_private\_endpoint) | Set this to true to force all api calls to use the IBM Cloud private endpoints. | `bool` | `true` | no |
 | <a name="input_verify_worker_network_readiness"></a> [verify\_worker\_network\_readiness](#input\_verify\_worker\_network\_readiness) | By setting this to true, a script will run kubectl commands to verify that all worker nodes can communicate successfully with the master. If the runtime does not have access to the kube cluster to run kubectl commands, this should be set to false. | `bool` | `true` | no |
@@ -106,6 +110,7 @@ The following resources are provisioned by this example:
 | <a name="output_cluster_name"></a> [cluster\_name](#output\_cluster\_name) | The name of the provisioned OpenShift cluster. |
 | <a name="output_cos_crn"></a> [cos\_crn](#output\_cos\_crn) | The Cloud Resource Name (CRN) of the Object Storage instance associated with the cluster. |
 | <a name="output_ingress_hostname"></a> [ingress\_hostname](#output\_ingress\_hostname) | The hostname assigned to the Cluster's Ingress subdomain for external access. |
+| <a name="output_ingress_secrets_group"></a> [ingress\_secrets\_group](#output\_ingress\_secrets\_group) | Info of secrets group for storing ingress certificates |
 | <a name="output_kms_config"></a> [kms\_config](#output\_kms\_config) | Configuration details for Key Management Service (KMS) used for encryption in the cluster. |
 | <a name="output_master_status"></a> [master\_status](#output\_master\_status) | The current status of the Kubernetes master node in the cluster. |
 | <a name="output_master_url"></a> [master\_url](#output\_master\_url) | The API endpoint URL for the Kubernetes master node of the cluster. |

solutions/fully-configurable/main.tf

@@ -230,4 +230,8 @@ module "ocp_base" {
   use_private_endpoint                  = var.use_private_endpoint
   verify_worker_network_readiness       = var.verify_worker_network_readiness
   worker_pools_taints                   = var.worker_pools_taints
+  enable_secrets_manager_for_ingress    = var.enable_secrets_manager_for_ingress
+  existing_secrets_manager_instance_crn = var.existing_secrets_manager_instance_crn
+  secrets_manager_secret_group_id       = var.secrets_manager_secret_group_id
+  is_default_secrets_manager_instance   = var.is_default_secrets_manager_instance
 }

solutions/fully-configurable/outputs.tf

@@ -101,3 +101,8 @@ output "registry_vpe" {
   description = "Details of the registry Virtual Private Endpoint (VPE), if it exists. For more info about schema, see https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/data-sources/is_virtual_endpoint_gateway"
   value       = module.ocp_base.registry_vpe
 }
+
+output "ingress_secrets_group" {
+  description = "Info of secrets group for storing ingress certificates"
+  value       = module.ocp_base.ingress_secrets_group
+}