feat: add variable enable_vpc_cni_prefix_delegation (#157)

<!--
  ~ Copyright 2023 StreamNative, Inc.
  ~
  ~ Licensed under the Apache License, Version 2.0 (the "License");
  ~ you may not use this file except in compliance with the License.
  ~ You may obtain a copy of the License at
  ~
  ~     http://www.apache.org/licenses/LICENSE-2.0
  ~
  ~ Unless required by applicable law or agreed to in writing, software
  ~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied.
  ~ See the License for the specific language governing permissions and
  ~ limitations under the License.
-->

<!--
### Contribution Checklist

- Fill out the template below to describe the changes contributed by the
pull request. That will give reviewers the context they need to do the
review.
  
- Each pull request should address only one issue, not mix up code from
multiple issues.
  
  - Each commit in the pull request has a meaningful commit message

- Once all items of the checklist are addressed, remove the above text
and this checklist, leaving only the filled out template below.

**(The sections below can be removed for hotfixes of typos)**
-->

### Motivation

In some cases, it's hard to use the prefix delegation. e.g, the Subnet
CIDR is very small, for /24 Subnet, the prefix delegation /28 can
conflict with ENI primary IP.

And the AWS requires the Subnet must have any contiguous /28 blocks
available. See
[here](https://docs.aws.amazon.com/eks/latest/userguide/cni-increase-ip-addresses-procedure.html#cni-increase-ip-procedure).

### Modifications

- Add a new variable `enable_vpc_cni_prefix_delegation`

### Verifying this change

- [ ] Make sure that the change passes the CI checks.

*(Please pick either of the following options)*

This change is a trivial rework / code cleanup without any test
coverage.

*(or)*

This change is already covered by existing tests, such as *(please
describe tests)*.

*(or)*

This change added tests and can be verified as follows:

*(example:)*
- *Added integration tests for end-to-end deployment with large payloads
(10MB)*
  - *Extended integration test for recovery after broker failure*

### Documentation

- [x] `doc` 
  
  (If this PR contains doc changes)

Signed-off-by: Max Xu <[email protected]>

Author: maxsxu

README.md

@@ -317,6 +317,7 @@ _Note: Since this module manages all of the Kubernetes addon dependencies requir
 | <a name="input_enable_v3_node_groups"></a> [enable\_v3\_node\_groups](#input\_enable\_v3\_node\_groups) | Enable v3 node groups, which uses a single ASG and all other node groups enabled elsewhere | `bool` | `false` | no |
 | <a name="input_enable_v3_node_migration"></a> [enable\_v3\_node\_migration](#input\_enable\_v3\_node\_migration) | Enable v3 node and v2 node groups at the same time. Intended for use with migration to v3 nodes. | `bool` | `false` | no |
 | <a name="input_enable_v3_node_taints"></a> [enable\_v3\_node\_taints](#input\_enable\_v3\_node\_taints) | When v3 node groups are enabled, use the node taints. Defaults to true | `bool` | `true` | no |
+| <a name="input_enable_vpc_cni_prefix_delegation"></a> [enable\_vpc\_cni\_prefix\_delegation](#input\_enable\_vpc\_cni\_prefix\_delegation) | Whether set ENABLE\_PREFIX\_DELEGATION for vpc-cni addon | `bool` | `true` | no |
 | <a name="input_external_dns_helm_chart_name"></a> [external\_dns\_helm\_chart\_name](#input\_external\_dns\_helm\_chart\_name) | The name of the Helm chart in the repository for ExternalDNS. | `string` | `"external-dns"` | no |
 | <a name="input_external_dns_helm_chart_repository"></a> [external\_dns\_helm\_chart\_repository](#input\_external\_dns\_helm\_chart\_repository) | The repository containing the ExternalDNS helm chart. | `string` | `"https://charts.bitnami.com/bitnami"` | no |
 | <a name="input_external_dns_helm_chart_version"></a> [external\_dns\_helm\_chart\_version](#input\_external\_dns\_helm\_chart\_version) | Helm chart version for ExternalDNS. See https://hub.helm.sh/charts/bitnami/external-dns for updates. | `string` | `"6.10.2"` | no |

main.tf

@@ -287,8 +287,8 @@ module "eks" {
       configuration_values = jsonencode({
         env = {
           # Reference docs https://docs.aws.amazon.com/eks/latest/userguide/cni-increase-ip-addresses.html
-          ENABLE_PREFIX_DELEGATION = "true"
-          WARM_PREFIX_TARGET       = "1"
+          ENABLE_PREFIX_DELEGATION = var.enable_vpc_cni_prefix_delegation ? "true" : null
+          WARM_PREFIX_TARGET       = var.enable_vpc_cni_prefix_delegation ? "1" : null
         }
       })
     }

variables.tf

@@ -747,3 +747,9 @@ variable "node_groups" {
   type        = any
   default     = null
 }
+
+variable "enable_vpc_cni_prefix_delegation" {
+  type        = bool
+  default     = true
+  description = "Whether set ENABLE_PREFIX_DELEGATION for vpc-cni addon"
+}