Skip to content

Commit 4241f5f

Browse files
bryantbiggsbryantbiggs
committed
fix: Remediate type mismatch for EFA interfaces and ensure correct (local) definition is used
1 parent f56e87e commit 4241f5f

File tree

8 files changed

+74
-32
lines changed

8 files changed

+74
-32
lines changed

‎modules/eks-managed-node-group/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,7 @@ module "eks_managed_node_group" {
169169
| <a name="input_metadata_options"></a> [metadata\_options](#input\_metadata\_options) | Customize the metadata options for the instance | <pre>object({<br/> http_endpoint = optional(string, "enabled")<br/> http_protocol_ipv6 = optional(string)<br/> http_put_response_hop_limit = optional(number, 1)<br/> http_tokens = optional(string, "required")<br/> instance_metadata_tags = optional(string)<br/> })</pre> | <pre>{<br/> "http_endpoint": "enabled",<br/> "http_put_response_hop_limit": 1,<br/> "http_tokens": "required"<br/>}</pre> | no |
170170
| <a name="input_min_size"></a> [min\_size](#input\_min\_size) | Minimum number of instances/nodes | `number` | `1` | no |
171171
| <a name="input_name"></a> [name](#input\_name) | Name of the EKS managed node group | `string` | `""` | no |
172-
| <a name="input_network_interfaces"></a> [network\_interfaces](#input\_network\_interfaces) | Customize network interfaces to be attached at instance boot time | <pre>list(object({<br/> associate_carrier_ip_address = optional(bool)<br/> associate_public_ip_address = optional(bool)<br/> connection_tracking_specification = optional(object({<br/> tcp_established_timeout = optional(number)<br/> udp_stream_timeout = optional(number)<br/> udp_timeout = optional(number)<br/> }))<br/> delete_on_termination = optional(bool)<br/> description = optional(string)<br/> device_index = optional(number)<br/> ena_srd_specification = optional(object({<br/> ena_srd_enabled = optional(bool)<br/> ena_srd_udp_specification = optional(object({<br/> ena_srd_udp_enabled = optional(bool)<br/> }))<br/> }))<br/> interface_type = optional(string)<br/> ipv4_address_count = optional(number)<br/> ipv4_addresses = optional(list(string))<br/> ipv4_prefix_count = optional(number)<br/> ipv4_prefixes = optional(list(string))<br/> ipv6_address_count = optional(number)<br/> ipv6_addresses = optional(list(string))<br/> ipv6_prefix_count = optional(number)<br/> ipv6_prefixes = optional(list(string))<br/> network_card_index = optional(number)<br/> network_interface_id = optional(string)<br/> primary_ipv6 = optional(bool)<br/> private_ip_address = optional(string)<br/> security_groups = optional(list(string), [])<br/> subnet_id = optional(string)<br/> }))</pre> | `[]` | no |
172+
| <a name="input_network_interfaces"></a> [network\_interfaces](#input\_network\_interfaces) | Customize network interfaces to be attached at instance boot time | <pre>list(object({<br/> associate_carrier_ip_address = optional(bool)<br/> associate_public_ip_address = optional(bool)<br/> connection_tracking_specification = optional(object({<br/> tcp_established_timeout = optional(number)<br/> udp_stream_timeout = optional(number)<br/> udp_timeout = optional(number)<br/> }))<br/> delete_on_termination = optional(bool)<br/> description = optional(string)<br/> device_index = optional(number)<br/> ena_srd_specification = optional(object({<br/> ena_srd_enabled = optional(bool)<br/> ena_srd_udp_specification = optional(object({<br/> ena_srd_udp_enabled = optional(bool)<br/> }))<br/> }))<br/> interface_type = optional(string)<br/> ipv4_address_count = optional(number)<br/> ipv4_addresses = optional(list(string))<br/> ipv4_prefix_count = optional(number)<br/> ipv4_prefixes = optional(list(string))<br/> ipv6_address_count = optional(number)<br/> ipv6_addresses = optional(list(string))<br/> ipv6_prefix_count = optional(number)<br/> ipv6_prefixes = optional(list(string))<br/> network_card_index = optional(number)<br/> network_interface_id = optional(string)<br/> primary_ipv6 = optional(bool)<br/> private_ip_address = optional(string)<br/> security_groups = optional(list(string), [])<br/> }))</pre> | `[]` | no |
173173
| <a name="input_node_repair_config"></a> [node\_repair\_config](#input\_node\_repair\_config) | The node auto repair configuration for the node group | <pre>object({<br/> enabled = optional(bool, true)<br/> })</pre> | `null` | no |
174174
| <a name="input_partition"></a> [partition](#input\_partition) | The AWS partition - pass through value to reduce number of GET requests from data sources | `string` | `""` | no |
175175
| <a name="input_placement"></a> [placement](#input\_placement) | The placement of the instance | <pre>object({<br/> affinity = optional(string)<br/> availability_zone = optional(string)<br/> group_name = optional(string)<br/> host_id = optional(string)<br/> host_resource_group_arn = optional(string)<br/> partition_number = optional(number)<br/> spread_domain = optional(string)<br/> tenancy = optional(string)<br/> })</pre> | `null` | no |

‎modules/eks-managed-node-group/main.tf‎

Lines changed: 38 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ module "user_data" {
4141
################################################################################
4242

4343
data "aws_ec2_instance_type" "this" {
44-
count = var.create && var.enable_efa_support ? 1 : 0
44+
count = local.enable_efa_support ? 1 : 0
4545

4646
region = var.region
4747

@@ -62,6 +62,24 @@ locals {
6262
device_index = i == 0 ? 0 : 1
6363
network_card_index = i
6464
interface_type = var.enable_efa_only ? contains(concat([0], var.efa_indices), i) ? "efa" : "efa-only" : "efa"
65+
66+
# Null out due to error: The true and false result expressions must have consistent types. The 'true' value is tuple, but the 'false' value is list of objects.
67+
associate_carrier_ip_address = null
68+
connection_tracking_specification = null
69+
description = "EFA${var.enable_efa_only ? "-only" : ""} Network Interface ${i}"
70+
ena_srd_specification = null
71+
ipv4_address_count = null
72+
ipv4_addresses = null
73+
ipv4_prefix_count = null
74+
ipv4_prefixes = null
75+
ipv6_address_count = null
76+
ipv6_addresses = null
77+
ipv6_prefix_count = null
78+
ipv6_prefixes = null
79+
network_interface_id = null
80+
primary_ipv6 = null
81+
private_ip_address = null
82+
security_groups = []
6583
}
6684
]
6785

@@ -247,7 +265,7 @@ resource "aws_launch_template" "this" {
247265
name_prefix = var.launch_template_use_name_prefix ? "${local.launch_template_name}-" : null
248266

249267
dynamic "network_interfaces" {
250-
for_each = length(var.network_interfaces) > 0 ? var.network_interfaces : []
268+
for_each = length(local.network_interfaces) > 0 ? local.network_interfaces : []
251269

252270
content {
253271
associate_carrier_ip_address = network_interfaces.value.associate_carrier_ip_address
@@ -688,9 +706,16 @@ locals {
688706
{
689707
all_self_efa = {
690708
description = "Node to node EFA"
691-
protocol = "-1"
692-
from_port = 0
709+
ip_protocol = "-1"
710+
from_port = null
693711
self = true
712+
713+
# Null out due to variable type and not using `try()` in resource
714+
cidr_ipv4 = null
715+
cidr_ipv6 = null
716+
name = null
717+
prefix_list_id = null
718+
tags = {}
694719
}
695720
} : k => v if var.enable_efa_support
696721
},
@@ -700,9 +725,16 @@ locals {
700725
{
701726
all_self_efa = {
702727
description = "Node to node EFA"
703-
protocol = "-1"
704-
to_port = 0
728+
ip_protocol = "-1"
729+
to_port = null
705730
self = true
731+
732+
# Null out due to variable type and not using `try()` in resource
733+
cidr_ipv4 = null
734+
cidr_ipv6 = null
735+
name = null
736+
prefix_list_id = null
737+
tags = {}
706738
}
707739
} : k => v if var.enable_efa_support
708740
},

‎modules/eks-managed-node-group/variables.tf‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -384,7 +384,6 @@ variable "network_interfaces" {
384384
primary_ipv6 = optional(bool)
385385
private_ip_address = optional(string)
386386
security_groups = optional(list(string), [])
387-
subnet_id = optional(string)
388387
}))
389388
default = []
390389
nullable = false

‎modules/self-managed-node-group/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,7 @@ module "self_managed_node_group" {
166166
| <a name="input_min_size"></a> [min\_size](#input\_min\_size) | The minimum size of the autoscaling group | `number` | `1` | no |
167167
| <a name="input_mixed_instances_policy"></a> [mixed\_instances\_policy](#input\_mixed\_instances\_policy) | Configuration block containing settings to define launch targets for Auto Scaling groups | <pre>object({<br/> instances_distribution = optional(object({<br/> on_demand_allocation_strategy = optional(string)<br/> on_demand_base_capacity = optional(number)<br/> on_demand_percentage_above_base_capacity = optional(number)<br/> spot_allocation_strategy = optional(string)<br/> spot_instance_pools = optional(number)<br/> spot_max_price = optional(string)<br/> }))<br/> launch_template = object({<br/> override = optional(list(object({<br/> instance_requirements = optional(object({<br/> accelerator_count = optional(object({<br/> max = optional(number)<br/> min = optional(number)<br/> }))<br/> accelerator_manufacturers = optional(list(string))<br/> accelerator_names = optional(list(string))<br/> accelerator_total_memory_mib = optional(object({<br/> max = optional(number)<br/> min = optional(number)<br/> }))<br/> accelerator_types = optional(list(string))<br/> allowed_instance_types = optional(list(string))<br/> bare_metal = optional(string)<br/> baseline_ebs_bandwidth_mbps = optional(object({<br/> max = optional(number)<br/> min = optional(number)<br/> }))<br/> burstable_performance = optional(string)<br/> cpu_manufacturers = optional(list(string))<br/> excluded_instance_types = optional(list(string))<br/> instance_generations = optional(list(string))<br/> local_storage = optional(string)<br/> local_storage_types = optional(list(string))<br/> max_spot_price_as_percentage_of_optimal_on_demand_price = optional(number)<br/> memory_gib_per_vcpu = optional(object({<br/> max = optional(number)<br/> min = optional(number)<br/> }))<br/> memory_mib = optional(object({<br/> max = optional(number)<br/> min = optional(number)<br/> }))<br/> network_bandwidth_gbps = optional(object({<br/> max = optional(number)<br/> min = optional(number)<br/> }))<br/> network_interface_count = optional(object({<br/> max = optional(number)<br/> min = optional(number)<br/> }))<br/> on_demand_max_price_percentage_over_lowest_price = optional(number)<br/> require_hibernate_support = optional(bool)<br/> spot_max_price_percentage_over_lowest_price = optional(number)<br/> total_local_storage_gb = optional(object({<br/> max = optional(number)<br/> min = optional(number)<br/> }))<br/> vcpu_count = optional(object({<br/> max = optional(number)<br/> min = optional(number)<br/> }))<br/> }))<br/> instance_type = optional(string)<br/> launch_template_specification = optional(object({<br/> launch_template_id = optional(string)<br/> launch_template_name = optional(string)<br/> version = optional(string)<br/> }))<br/> weighted_capacity = optional(string)<br/> })))<br/> })<br/> })</pre> | `null` | no |
168168
| <a name="input_name"></a> [name](#input\_name) | Name of the Self managed Node Group | `string` | `""` | no |
169-
| <a name="input_network_interfaces"></a> [network\_interfaces](#input\_network\_interfaces) | Customize network interfaces to be attached at instance boot time | <pre>list(object({<br/> associate_carrier_ip_address = optional(bool)<br/> associate_public_ip_address = optional(bool)<br/> connection_tracking_specification = optional(object({<br/> tcp_established_timeout = optional(number)<br/> udp_stream_timeout = optional(number)<br/> udp_timeout = optional(number)<br/> }))<br/> delete_on_termination = optional(bool)<br/> description = optional(string)<br/> device_index = optional(number)<br/> ena_srd_specification = optional(object({<br/> ena_srd_enabled = optional(bool)<br/> ena_srd_udp_specification = optional(object({<br/> ena_srd_udp_enabled = optional(bool)<br/> }))<br/> }))<br/> interface_type = optional(string)<br/> ipv4_address_count = optional(number)<br/> ipv4_addresses = optional(list(string))<br/> ipv4_prefix_count = optional(number)<br/> ipv4_prefixes = optional(list(string))<br/> ipv6_address_count = optional(number)<br/> ipv6_addresses = optional(list(string))<br/> ipv6_prefix_count = optional(number)<br/> ipv6_prefixes = optional(list(string))<br/> network_card_index = optional(number)<br/> network_interface_id = optional(string)<br/> primary_ipv6 = optional(bool)<br/> private_ip_address = optional(string)<br/> security_groups = optional(list(string), [])<br/> subnet_id = optional(string)<br/> }))</pre> | `[]` | no |
169+
| <a name="input_network_interfaces"></a> [network\_interfaces](#input\_network\_interfaces) | Customize network interfaces to be attached at instance boot time | <pre>list(object({<br/> associate_carrier_ip_address = optional(bool)<br/> associate_public_ip_address = optional(bool)<br/> connection_tracking_specification = optional(object({<br/> tcp_established_timeout = optional(number)<br/> udp_stream_timeout = optional(number)<br/> udp_timeout = optional(number)<br/> }))<br/> delete_on_termination = optional(bool)<br/> description = optional(string)<br/> device_index = optional(number)<br/> ena_srd_specification = optional(object({<br/> ena_srd_enabled = optional(bool)<br/> ena_srd_udp_specification = optional(object({<br/> ena_srd_udp_enabled = optional(bool)<br/> }))<br/> }))<br/> interface_type = optional(string)<br/> ipv4_address_count = optional(number)<br/> ipv4_addresses = optional(list(string))<br/> ipv4_prefix_count = optional(number)<br/> ipv4_prefixes = optional(list(string))<br/> ipv6_address_count = optional(number)<br/> ipv6_addresses = optional(list(string))<br/> ipv6_prefix_count = optional(number)<br/> ipv6_prefixes = optional(list(string))<br/> network_card_index = optional(number)<br/> network_interface_id = optional(string)<br/> primary_ipv6 = optional(bool)<br/> private_ip_address = optional(string)<br/> security_groups = optional(list(string), [])<br/> }))</pre> | `[]` | no |
170170
| <a name="input_partition"></a> [partition](#input\_partition) | The AWS partition - pass through value to reduce number of GET requests from data sources | `string` | `""` | no |
171171
| <a name="input_placement"></a> [placement](#input\_placement) | The placement of the instance | <pre>object({<br/> affinity = optional(string)<br/> availability_zone = optional(string)<br/> group_name = optional(string)<br/> host_id = optional(string)<br/> host_resource_group_arn = optional(string)<br/> partition_number = optional(number)<br/> spread_domain = optional(string)<br/> tenancy = optional(string)<br/> })</pre> | `null` | no |
172172
| <a name="input_placement_group"></a> [placement\_group](#input\_placement\_group) | The name of the placement group into which you'll launch your instances | `string` | `null` | no |

‎modules/self-managed-node-group/main.tf‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ module "user_data" {
8282
################################################################################
8383

8484
data "aws_ec2_instance_type" "this" {
85-
count = var.create && var.enable_efa_support ? 1 : 0
85+
count = local.enable_efa_support ? 1 : 0
8686

8787
region = var.region
8888

@@ -103,6 +103,24 @@ locals {
103103
device_index = i == 0 ? 0 : 1
104104
network_card_index = i
105105
interface_type = var.enable_efa_only ? contains(concat([0], var.efa_indices), i) ? "efa" : "efa-only" : "efa"
106+
107+
# Null out due to error: The true and false result expressions must have consistent types. The 'true' value is tuple, but the 'false' value is list of objects.
108+
associate_carrier_ip_address = null
109+
connection_tracking_specification = null
110+
description = "EFA${var.enable_efa_only ? "-only" : ""} Network Interface ${i}"
111+
ena_srd_specification = null
112+
ipv4_address_count = null
113+
ipv4_addresses = null
114+
ipv4_prefix_count = null
115+
ipv4_prefixes = null
116+
ipv6_address_count = null
117+
ipv6_addresses = null
118+
ipv6_prefix_count = null
119+
ipv6_prefixes = null
120+
network_interface_id = null
121+
primary_ipv6 = null
122+
private_ip_address = null
123+
security_groups = []
106124
}
107125
]
108126

@@ -365,7 +383,7 @@ resource "aws_launch_template" "this" {
365383
name_prefix = var.launch_template_use_name_prefix ? "${local.launch_template_name}-" : null
366384

367385
dynamic "network_interfaces" {
368-
for_each = length(var.network_interfaces) > 0 ? var.network_interfaces : []
386+
for_each = length(local.network_interfaces) > 0 ? local.network_interfaces : []
369387

370388
content {
371389
associate_carrier_ip_address = network_interfaces.value.associate_carrier_ip_address

‎modules/self-managed-node-group/variables.tf‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -311,7 +311,6 @@ variable "network_interfaces" {
311311
primary_ipv6 = optional(bool)
312312
private_ip_address = optional(string)
313313
security_groups = optional(list(string), [])
314-
subnet_id = optional(string)
315314
}))
316315
default = []
317316
nullable = false

‎tests/eks-managed-node-group/main.tf‎

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -318,14 +318,15 @@ module "eks" {
318318
}
319319

320320
efa = {
321-
# Disabling automatic creation due to instance type/quota availability
322-
# Can be enabled when appropriate for testing/validation
323-
create = false
324-
325321
# The EKS AL2023 NVIDIA AMI provides all of the necessary components
326322
# for accelerated workloads w/ EFA
327323
ami_type = "AL2023_x86_64_NVIDIA"
328-
instance_types = ["p5e.48xlarge"]
324+
instance_types = ["p4d.24xlarge"]
325+
326+
# Setting to zero so all resources are created *EXCEPT the EC2 instances
327+
min_size = 0
328+
max_size = 1
329+
desired_size = 0
329330

330331
# Mount instance store volumes in RAID-0 for kubelet and containerd
331332
# https://github.com/awslabs/amazon-eks-ami/blob/master/doc/USER_GUIDE.md#raid-0-for-kubelet-and-containerd-raid0
@@ -350,11 +351,7 @@ module "eks" {
350351
# 3. Expose all of the available EFA interfaces on the launch template
351352
enable_efa_support = true
352353
enable_efa_only = true
353-
efa_indices = [0, 4, 8, 12]
354-
355-
min_size = 1
356-
max_size = 1
357-
desired_size = 1
354+
efa_indices = [0]
358355

359356
labels = {
360357
"vpc.amazonaws.com/efa.present" = "true"

0 commit comments

Comments
 (0)