cluster_addons before_compute does not guarantee addons are created before compute resources

[vincentmrg]

Description

The cluster_addons feature in the module provides a before_compute attribute, which is described as a way to ensure that certain addons are created before compute resources (managed by module.eks_managed_node_group / module.self_managed_node_group).

In practice, however, this attribute does not guarantee that addons are created before compute resources. Instead, it only ensures that compute resources are created before addons with before_compute = false

terraform-aws-eks/main.tf

Line 801 in de2aa10

module.eks_managed_node_group,

This behavior occurs because the modules responsible for creating compute resources currently do not have a depends_on referencing the addons with before_compute = true.

• ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version: 21.3.1

Expected behavior

Addons with before_compute = true should be applied before any compute resources.

The modules module.eks_managed_node_group and module.self_managed_node_group should include a depends_on on the aws_eks_addon.before_compute resources.

Terraform documentation on depends_on for modules.

[bryantbiggs]

please describe the underlying problem you are facing (i.e. - "I did this, and expected this, but got this")

[vincentmrg]

I deployed an EKS cluster with somecluster_addons configured with before_compute = true. I expected them to be created before compute resources, they were not.

[bryantbiggs]

well, that will depend. hard to say without reproductions

[vincentmrg]

You're right, it's difficult to reproduce.

For my part, this happens sometimes when applying my infrastructure as code; it's not systematic depending on the order in which the resources are created.

I'm not going to share my entire setup here, but with a cluster_addons setup like this:

cluster_addons = {
    coredns    = {}
    kube-proxy = {}
    vpc-cni = {
      # CNI should be installed before worker nodes to prevent the following error on nodes:
      # KubeletNotReady              container runtime network not ready: NetworkReady=false reason: NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
      before_compute = true
    }
  }

I have a failure to apply my plan on node groups creation like this:

module.platform.module.eks.module.eks_managed_node_group["system"].aws_eks_node_group.this[0]: Still creating... [33m00s elapsed]
module.platform.module.eks.module.eks_managed_node_group["system"].aws_eks_node_group.this[0]: Still creating... [33m10s elapsed]
module.platform.module.eks.module.eks_managed_node_group["system"].aws_eks_node_group.this[0]: Still creating... [33m20s elapsed]
module.platform.module.eks.module.eks_managed_node_group["system"].aws_eks_node_group.this[0]: Still creating... [33m30s elapsed]
╷
│ Error: waiting for EKS Node Group (dataplane-staging-us-east-2-01:system-20250925085111346800000038) create: unexpected state 'CREATE_FAILED', wanted target 'ACTIVE'. last error: i-003a1046d6277a5a8, i-0840bf7d11e0caf25: NodeCreationFailure: Unhealthy nodes in the kubernetes cluster
│ 
│   with module.platform.module.eks.module.eks_managed_node_group["system"].aws_eks_node_group.this[0],
│   on .terraform/modules/platform.eks/modules/eks-managed-node-group/main.tf line 395, in resource "aws_eks_node_group" "this":
│  395: resource "aws_eks_node_group" "this" {

As I explained above, this would definitely be fixed by adding the correct dependencies to the modules responsible for creating nodegroups.
A PR making these changes has been opened here: #3529

[CrazyFunker]

We just ran into a problem related to this. We're using eks_managed_node_group as well.
Node groups were failing health checks because the AWS VPC CNI and KubeProxy addons were not created before adding instances to the node group. It is easily reproducible on any new EKS cluster you create for any version of the module.
One of the symptoms to this is the cloudTrail error: Value (eks-72cccab3-dfde-7bc8-a8c6-516de7beeb6f) for parameter iamInstanceProfile.name is invalid. Invalid IAM Instance Profile name

[bryantbiggs]

We just ran into a problem related to this. We're using eks_managed_node_group as well.

Talk is cheap, show the reproduction code 😅

It is easily reproducible on any new EKS cluster you create for any version of the module.

If that were true, we'd see a whole bunch of reports on this

[CrazyFunker]

We just ran into a problem related to this. We're using eks_managed_node_group as well. Node groups were failing health checks because the AWS VPC CNI and KubeProxy addons were not created before adding instances to the node group. It is easily reproducible on any new EKS cluster you create for any version of the module. One of the symptoms to this is the cloudTrail error: Value (eks-72cccab3-dfde-7bc8-a8c6-516de7beeb6f) for parameter iamInstanceProfile.name is invalid. Invalid IAM Instance Profile name

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "21.2.0"

  name                   = local.eks_cluster_name
  kubernetes_version     = local.eks_cluster_version
  enabled_log_types      = local.eks_cluster_enabled_log_types
  endpoint_public_access = local.eks_cluster_endpoint_public_access

  tags = {
    env               = var.env_name
    "cluster:name"    = local.eks_cluster_name
    "cluster:version" = local.eks_cluster_version
  }
  addons = {
    coredns = {
      most_recent = true
    }
    kube-proxy = {
      most_recent = true
    }
    vpc-cni = {
      most_recent = true
    }
    eks-pod-identity-agent = {
      most_recent = true
    }
    aws-ebs-csi-driver = {
      service_account_role_arn = aws_iam_role.eks_ebs_csi_driver.arn
      most_recent              = true
    }
    aws-efs-csi-driver = {
      service_account_role_arn = aws_iam_role.eks_efs_csi_driver.arn
      most_recent              = true
    }
    snapshot-controller = {
      most_recent = true
    }
    eks-node-monitoring-agent = {
      most_recent = true
    }
  }

  vpc_id                                   = module.vpc.vpc_id
  subnet_ids                               = module.vpc.private_subnets
  control_plane_subnet_ids                 = module.vpc.intra_subnets
  enable_cluster_creator_admin_permissions = true

  eks_managed_node_groups = {
    default = {
      ami_type                       = "BOTTLEROCKET_x86_64"
      use_latest_ami_release_version = true
      instance_types = ["t3a.medium"]

      metadata_options = {
        http_endpoint               = "enabled"
        http_tokens                 = "required"
        http_put_response_hop_limit = 1
      }

      block_device_mappings = {
        xvda = {
          device_name = "/dev/xvda"
          ebs = {
            volume_size           = 8
            volume_type           = "gp3"
            iops                  = 3000
            throughput            = 125
            encrypted             = true
            delete_on_termination = true
          }
        }
        xvdb = {
          device_name = "/dev/xvdb"
          ebs = {
            volume_size           = 100
            volume_type           = "gp3"
            iops                  = 3000
            throughput            = 125
            encrypted             = true
            delete_on_termination = true
          }
        }
      }

      min_size     = 3
      max_size     = 3
      desired_size = 3

      node_repair_config = {
        enabled = true
      }

      launch_template_tags = {
        Name = "${local.eks_cluster_name}-eks-managed-node"
        env  = var.env_name
      }
      # capacity_type = "ON_DEMAND"
      update_config = {
        # Using fixed count instead of percentage to ensure predictable update behavior with fixed node group size
        max_unavailable = 1
      }
      labels = {
        "env"                    = var.env_name
        "cluster-name"           = local.eks_cluster_name
        "cluster-version"        = local.eks_cluster_version
        "k8s.xxx.com/nodepool" = "critical"
      }

      taints = {
        critical = {
          key    = "CriticalAddonsOnly"
          value  = "true"
          effect = "NO_SCHEDULE"
        }
      }
    }
    argocd = {
      ami_type                       = "BOTTLEROCKET_x86_64"
      use_latest_ami_release_version = true
      instance_types = ["m6a.xlarge"]

      metadata_options = {
        http_endpoint               = "enabled"
        http_tokens                 = "required"
        http_put_response_hop_limit = 1
      }

      block_device_mappings = {
        xvda = {
          device_name = "/dev/xvda"
          ebs = {
            volume_size           = 8
            volume_type           = "gp3"
            iops                  = 3000
            throughput            = 125
            encrypted             = true
            delete_on_termination = true
          }
        }
        xvdb = {
          device_name = "/dev/xvdb"
          ebs = {
            volume_size           = 100
            volume_type           = "gp3"
            iops                  = 3000
            throughput            = 125
            encrypted             = true
            delete_on_termination = true
          }
        }
      }
      min_size     = 3
      max_size     = 3
      desired_size = 3

      node_repair_config = {
        enabled = true
      }

      launch_template_tags = {
        Name = "${local.eks_cluster_name}-eks-managed-node-argocd"
        env  = var.env_name
      }
      update_config = {
        max_unavailable = 1
      }
      labels = {
        "env"                    = var.env_name
        "cluster-name"           = local.eks_cluster_name
        "cluster-version"        = local.eks_cluster_version
        "k8s.xxx.com/nodepool" = "argocd"
      }
    }
  }

  node_security_group_tags = {
    "karpenter.sh/discovery" = "${local.eks_cluster_name}-node"
  }
}

[bryantbiggs]

1. Thats not deployable
2. You are not setting before_compute = true