Skip to content

Commit 7c99bb1

Browse files
bryantbiggsbryantbiggs
authored
feat: Add support for AL2023 nodeadm user data (#2942)
1 parent e6c3e90 commit 7c99bb1

File tree

18 files changed

+599
-38
lines changed

18 files changed

+599
-38
lines changed

‎docs/UPGRADE-20.0.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ To give users advanced notice and provide some future direction for this module,
2020
1. The `aws-auth` sub-module will be removed entirely from the project. Since this sub-module is captured in the v20.x releases, users can continue using it even after the module moves forward with the next major version. The long term strategy and direction is cluster access entry and to rely only on the AWS Terraform provider.
2121
2. The default value for `authentication_mode` will change to `API`. Aligning with point 1 above, this is a one way change, but users are free to specify the value of their choosing in place of this default (when the change is made). This module will proceed with an EKS API first strategy.
2222
3. The launch template and autoscaling group usage contained within the EKS managed nodegroup and self-managed nodegroup sub-modules *might be replaced with the [`terraform-aws-autoscaling`](https://github.com/terraform-aws-modules/terraform-aws-autoscaling) module. At minimum, it makes sense to replace most of functionality in the self-managed nodegroup module with this external module, but its not yet clear if there is any benefit of using it in the EKS managed nodegroup sub-module. The interface that users interact with will stay the same, the changes will be internal to the implementation and we will do everything we can to keep the disruption to a minimum.
23+
4. The `platform` variable will be replaced and instead `ami_type` will become the standard across both self-managed nodegroup(s) and EKS managed nodegroup(s). As EKS expands its portfolio of supported operating systems, the `ami_type` is better suited to associate the correct user data format to the respective OS. The `platform` variable is a legacy artifact of self-managed nodegroups but not as descriptive as the `ami_type`, and therefore it will be removed in favor of `ami_type`.
2324

2425
## Additional changes
2526

‎examples/eks_managed_node_group/main.tf

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,29 @@ module "eks" {
8686
}
8787
}
8888

89+
# AL2023 node group utilizing new user data format which utilizes nodeadm
90+
# to join nodes to the cluster (instead of /etc/eks/bootstrap.sh)
91+
al2023_nodeadm = {
92+
platform = "al2023"
93+
94+
cloudinit_pre_nodeadm = [
95+
{
96+
content_type = "application/node.eks.aws"
97+
content = <<-EOT
98+
---
99+
apiVersion: node.eks.aws/v1alpha
100+
kind: NodeConfig
101+
spec:
102+
kubelet:
103+
config:
104+
shutdownGracePeriod: 30s
105+
featureGates:
106+
DisableKubeletCloudCredentialProviders: true
107+
EOT
108+
}
109+
]
110+
}
111+
89112
# Default node group - as provided by AWS EKS using Bottlerocket
90113
bottlerocket_default = {
91114
# By default, the module creates a launch template to ensure tags are propagated to instances, etc.,

‎examples/self_managed_node_group/main.tf

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,29 @@ module "eks" {
7272
# Default node group - as provisioned by the module defaults
7373
default_node_group = {}
7474

75+
# AL2023 node group utilizing new user data format which utilizes nodeadm
76+
# to join nodes to the cluster (instead of /etc/eks/bootstrap.sh)
77+
al2023_nodeadm = {
78+
platform = "al2023"
79+
80+
cloudinit_pre_nodeadm = [
81+
{
82+
content_type = "application/node.eks.aws"
83+
content = <<-EOT
84+
---
85+
apiVersion: node.eks.aws/v1alpha
86+
kind: NodeConfig
87+
spec:
88+
kubelet:
89+
config:
90+
shutdownGracePeriod: 30s
91+
featureGates:
92+
DisableKubeletCloudCredentialProviders: true
93+
EOT
94+
}
95+
]
96+
}
97+
7598
# Bottlerocket node group
7699
bottlerocket = {
77100
name = "bottlerocket-self-mng"

‎examples/user_data/README.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,10 @@ $ terraform apply
3030

3131
| Name | Source | Version |
3232
|------|--------|---------|
33+
| <a name="module_eks_mng_al2023_additional"></a> [eks\_mng\_al2023\_additional](#module\_eks\_mng\_al2023\_additional) | ../../modules/_user_data | n/a |
34+
| <a name="module_eks_mng_al2023_custom_ami"></a> [eks\_mng\_al2023\_custom\_ami](#module\_eks\_mng\_al2023\_custom\_ami) | ../../modules/_user_data | n/a |
35+
| <a name="module_eks_mng_al2023_custom_template"></a> [eks\_mng\_al2023\_custom\_template](#module\_eks\_mng\_al2023\_custom\_template) | ../../modules/_user_data | n/a |
36+
| <a name="module_eks_mng_al2023_no_op"></a> [eks\_mng\_al2023\_no\_op](#module\_eks\_mng\_al2023\_no\_op) | ../../modules/_user_data | n/a |
3337
| <a name="module_eks_mng_al2_additional"></a> [eks\_mng\_al2\_additional](#module\_eks\_mng\_al2\_additional) | ../../modules/_user_data | n/a |
3438
| <a name="module_eks_mng_al2_custom_ami"></a> [eks\_mng\_al2\_custom\_ami](#module\_eks\_mng\_al2\_custom\_ami) | ../../modules/_user_data | n/a |
3539
| <a name="module_eks_mng_al2_custom_template"></a> [eks\_mng\_al2\_custom\_template](#module\_eks\_mng\_al2\_custom\_template) | ../../modules/_user_data | n/a |
@@ -42,6 +46,9 @@ $ terraform apply
4246
| <a name="module_eks_mng_windows_custom_ami"></a> [eks\_mng\_windows\_custom\_ami](#module\_eks\_mng\_windows\_custom\_ami) | ../../modules/_user_data | n/a |
4347
| <a name="module_eks_mng_windows_custom_template"></a> [eks\_mng\_windows\_custom\_template](#module\_eks\_mng\_windows\_custom\_template) | ../../modules/_user_data | n/a |
4448
| <a name="module_eks_mng_windows_no_op"></a> [eks\_mng\_windows\_no\_op](#module\_eks\_mng\_windows\_no\_op) | ../../modules/_user_data | n/a |
49+
| <a name="module_self_mng_al2023_bootstrap"></a> [self\_mng\_al2023\_bootstrap](#module\_self\_mng\_al2023\_bootstrap) | ../../modules/_user_data | n/a |
50+
| <a name="module_self_mng_al2023_custom_template"></a> [self\_mng\_al2023\_custom\_template](#module\_self\_mng\_al2023\_custom\_template) | ../../modules/_user_data | n/a |
51+
| <a name="module_self_mng_al2023_no_op"></a> [self\_mng\_al2023\_no\_op](#module\_self\_mng\_al2023\_no\_op) | ../../modules/_user_data | n/a |
4552
| <a name="module_self_mng_al2_bootstrap"></a> [self\_mng\_al2\_bootstrap](#module\_self\_mng\_al2\_bootstrap) | ../../modules/_user_data | n/a |
4653
| <a name="module_self_mng_al2_custom_template"></a> [self\_mng\_al2\_custom\_template](#module\_self\_mng\_al2\_custom\_template) | ../../modules/_user_data | n/a |
4754
| <a name="module_self_mng_al2_no_op"></a> [self\_mng\_al2\_no\_op](#module\_self\_mng\_al2\_no\_op) | ../../modules/_user_data | n/a |
@@ -56,6 +63,10 @@ $ terraform apply
5663

5764
| Name | Type |
5865
|------|------|
66+
| [local_file.eks_mng_al2023_additional](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
67+
| [local_file.eks_mng_al2023_custom_ami](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
68+
| [local_file.eks_mng_al2023_custom_template](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
69+
| [local_file.eks_mng_al2023_no_op](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
5970
| [local_file.eks_mng_al2_additional](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
6071
| [local_file.eks_mng_al2_custom_ami](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
6172
| [local_file.eks_mng_al2_custom_template](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
@@ -68,6 +79,9 @@ $ terraform apply
6879
| [local_file.eks_mng_windows_custom_ami](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
6980
| [local_file.eks_mng_windows_custom_template](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
7081
| [local_file.eks_mng_windows_no_op](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
82+
| [local_file.self_mng_al2023_bootstrap](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
83+
| [local_file.self_mng_al2023_custom_template](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
84+
| [local_file.self_mng_al2023_no_op](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
7185
| [local_file.self_mng_al2_bootstrap](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
7286
| [local_file.self_mng_al2_custom_template](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
7387
| [local_file.self_mng_al2_no_op](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |

‎examples/user_data/main.tf

Lines changed: 197 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,6 @@ module "eks_mng_al2_custom_ami" {
4343
EOT
4444
}
4545

46-
4746
module "eks_mng_al2_custom_template" {
4847
source = "../../modules/_user_data"
4948

@@ -65,6 +64,107 @@ module "eks_mng_al2_custom_template" {
6564
EOT
6665
}
6766

67+
################################################################################
68+
# EKS managed node group - AL2023
69+
################################################################################
70+
71+
module "eks_mng_al2023_no_op" {
72+
source = "../../modules/_user_data"
73+
74+
platform = "al2023"
75+
}
76+
77+
module "eks_mng_al2023_additional" {
78+
source = "../../modules/_user_data"
79+
80+
platform = "al2023"
81+
82+
cloudinit_pre_nodeadm = [{
83+
content = <<-EOT
84+
---
85+
apiVersion: node.eks.aws/v1alpha
86+
kind: NodeConfig
87+
spec:
88+
kubelet:
89+
config:
90+
shutdownGracePeriod: 30s
91+
featureGates:
92+
DisableKubeletCloudCredentialProviders: true
93+
EOT
94+
content_type = "application/node.eks.aws"
95+
}]
96+
}
97+
98+
module "eks_mng_al2023_custom_ami" {
99+
source = "../../modules/_user_data"
100+
101+
platform = "al2023"
102+
103+
cluster_name = local.name
104+
cluster_endpoint = local.cluster_endpoint
105+
cluster_auth_base64 = local.cluster_auth_base64
106+
cluster_service_ipv4_cidr = local.cluster_service_ipv4_cidr
107+
108+
enable_bootstrap_user_data = true
109+
110+
cloudinit_pre_nodeadm = [{
111+
content = <<-EOT
112+
---
113+
apiVersion: node.eks.aws/v1alpha
114+
kind: NodeConfig
115+
spec:
116+
kubelet:
117+
config:
118+
shutdownGracePeriod: 30s
119+
featureGates:
120+
DisableKubeletCloudCredentialProviders: true
121+
EOT
122+
content_type = "application/node.eks.aws"
123+
}]
124+
125+
cloudinit_post_nodeadm = [{
126+
content = <<-EOT
127+
echo "All done"
128+
EOT
129+
content_type = "text/x-shellscript; charset=\"us-ascii\""
130+
}]
131+
}
132+
133+
module "eks_mng_al2023_custom_template" {
134+
source = "../../modules/_user_data"
135+
136+
platform = "al2023"
137+
138+
cluster_name = local.name
139+
cluster_endpoint = local.cluster_endpoint
140+
cluster_auth_base64 = local.cluster_auth_base64
141+
142+
enable_bootstrap_user_data = true
143+
user_data_template_path = "${path.module}/templates/al2023_custom.tpl"
144+
145+
cloudinit_pre_nodeadm = [{
146+
content = <<-EOT
147+
---
148+
apiVersion: node.eks.aws/v1alpha
149+
kind: NodeConfig
150+
spec:
151+
kubelet:
152+
config:
153+
shutdownGracePeriod: 30s
154+
featureGates:
155+
DisableKubeletCloudCredentialProviders: true
156+
EOT
157+
content_type = "application/node.eks.aws"
158+
}]
159+
160+
cloudinit_post_nodeadm = [{
161+
content = <<-EOT
162+
echo "All done"
163+
EOT
164+
content_type = "text/x-shellscript; charset=\"us-ascii\""
165+
}]
166+
}
167+
68168
################################################################################
69169
# EKS managed node group - Bottlerocket
70170
################################################################################
@@ -80,6 +180,9 @@ module "eks_mng_bottlerocket_additional" {
80180

81181
platform = "bottlerocket"
82182

183+
# Should do nothing
184+
cluster_service_ipv4_cidr = local.cluster_service_ipv4_cidr
185+
83186
bootstrap_extra_args = <<-EOT
84187
# extra args added
85188
[settings.kernel]
@@ -138,6 +241,9 @@ module "eks_mng_windows_additional" {
138241

139242
platform = "windows"
140243

244+
# Should do nothing
245+
cluster_service_ipv4_cidr = local.cluster_service_ipv4_cidr
246+
141247
pre_bootstrap_user_data = <<-EOT
142248
[string]$Something = 'IDoNotKnowAnyPowerShell ¯\_(ツ)_/¯'
143249
EOT
@@ -245,6 +351,90 @@ module "self_mng_al2_custom_template" {
245351
EOT
246352
}
247353

354+
################################################################################
355+
# Self-managed node group - AL2023
356+
################################################################################
357+
358+
module "self_mng_al2023_no_op" {
359+
source = "../../modules/_user_data"
360+
361+
platform = "al2023"
362+
363+
is_eks_managed_node_group = false
364+
}
365+
366+
module "self_mng_al2023_bootstrap" {
367+
source = "../../modules/_user_data"
368+
369+
platform = "al2023"
370+
371+
enable_bootstrap_user_data = true
372+
is_eks_managed_node_group = false
373+
374+
cluster_name = local.name
375+
cluster_endpoint = local.cluster_endpoint
376+
cluster_auth_base64 = local.cluster_auth_base64
377+
378+
cloudinit_pre_nodeadm = [{
379+
content = <<-EOT
380+
---
381+
apiVersion: node.eks.aws/v1alpha
382+
kind: NodeConfig
383+
spec:
384+
kubelet:
385+
config:
386+
shutdownGracePeriod: 30s
387+
featureGates:
388+
DisableKubeletCloudCredentialProviders: true
389+
EOT
390+
content_type = "application/node.eks.aws"
391+
}]
392+
393+
cloudinit_post_nodeadm = [{
394+
content = <<-EOT
395+
echo "All done"
396+
EOT
397+
content_type = "text/x-shellscript; charset=\"us-ascii\""
398+
}]
399+
}
400+
401+
module "self_mng_al2023_custom_template" {
402+
source = "../../modules/_user_data"
403+
404+
platform = "al2023"
405+
406+
enable_bootstrap_user_data = true
407+
is_eks_managed_node_group = false
408+
409+
cluster_name = local.name
410+
cluster_endpoint = local.cluster_endpoint
411+
cluster_auth_base64 = local.cluster_auth_base64
412+
413+
user_data_template_path = "${path.module}/templates/al2023_custom.tpl"
414+
415+
cloudinit_pre_nodeadm = [{
416+
content = <<-EOT
417+
---
418+
apiVersion: node.eks.aws/v1alpha
419+
kind: NodeConfig
420+
spec:
421+
kubelet:
422+
config:
423+
shutdownGracePeriod: 30s
424+
featureGates:
425+
DisableKubeletCloudCredentialProviders: true
426+
EOT
427+
content_type = "application/node.eks.aws"
428+
}]
429+
430+
cloudinit_post_nodeadm = [{
431+
content = <<-EOT
432+
echo "All done"
433+
EOT
434+
content_type = "text/x-shellscript; charset=\"us-ascii\""
435+
}]
436+
}
437+
248438
################################################################################
249439
# Self-managed node group - Bottlerocket
250440
################################################################################
@@ -269,6 +459,9 @@ module "self_mng_bottlerocket_bootstrap" {
269459
cluster_endpoint = local.cluster_endpoint
270460
cluster_auth_base64 = local.cluster_auth_base64
271461

462+
# Should do nothing
463+
cluster_service_ipv4_cidr = local.cluster_service_ipv4_cidr
464+
272465
bootstrap_extra_args = <<-EOT
273466
# extra args added
274467
[settings.kernel]
@@ -321,6 +514,9 @@ module "self_mng_windows_bootstrap" {
321514
cluster_endpoint = local.cluster_endpoint
322515
cluster_auth_base64 = local.cluster_auth_base64
323516

517+
# Should do nothing
518+
cluster_service_ipv4_cidr = local.cluster_service_ipv4_cidr
519+
324520
pre_bootstrap_user_data = <<-EOT
325521
[string]$Something = 'IDoNotKnowAnyPowerShell ¯\_(ツ)_/¯'
326522
EOT

0 commit comments

Comments
 (0)