Skip to content

Karpenter Controller Policy LimitExceededΒ #3512

@jblackburn22

Description

@jblackburn22

Description

The Karpenter module version 21.2.0 generated the following error:

β•·
β”‚ Error: updating IAM Policy (arn:aws-us-gov:iam::redacted:policy/KarpenterController-20250908135531617300000016): operation error IAM: CreatePolicyVersion, https response error StatusCode: 409, RequestID: c0918f61-49b0-4ed0-9cad-65e105f11a41, LimitExceeded: Cannot exceed quota for PolicySize: 6144
β”‚
β”‚ with module.karpenter.aws_iam_policy.controller[0],
β”‚ on .terraform/modules/karpenter/modules/karpenter/main.tf line 65, in resource "aws_iam_policy" "controller":
β”‚ 65: resource "aws_iam_policy" "controller" {

The plan output is

module.karpenter.aws_iam_policy.controller[0] will be updated in-place

~ resource "aws_iam_policy" "controller" {
id = "arn:aws-us-gov:iam::redacted:policy/KarpenterController-20250908135531617300000016"
name = "KarpenterController-20250908135531617300000016"
~ policy = jsonencode(
~ {
~ Statement = [
# (1 unchanged element hidden)
{
Action = [
"ec2:RunInstances",
"ec2:CreateFleet",
]
Condition = {
StringEquals = {
"aws:ResourceTag/kubernetes.io/cluster/itsd-eks-cluster-dev" = "owned"
}
StringLike = {
"aws:ResourceTag/karpenter.sh/nodepool" = ""
}
}
Effect = "Allow"
Resource = "arn:aws-us-gov:ec2:us-gov-west-1:
:launch-template/"
Sid = "AllowScopedEC2LaunchTemplateAccessActions"
},
~ {
~ Resource = [
# (5 unchanged elements hidden)
"arn:aws-us-gov:ec2:us-gov-west-1:
:fleet/",
+ "arn:aws-us-gov:ec2:us-gov-west-1:
:capacity-reservation/",
]
# (4 unchanged attributes hidden)
},
{
Action = "ec2:CreateTags"
Condition = {
StringEquals = {
"aws:RequestTag/eks:eks-cluster-name" = "itsd-eks-cluster-dev"
"aws:RequestTag/kubernetes.io/cluster/itsd-eks-cluster-dev" = "owned"
"ec2:CreateAction" = [
"RunInstances",
"CreateFleet",
"CreateLaunchTemplate",
]
}
StringLike = {
"aws:RequestTag/karpenter.sh/nodepool" = "
"
}
}
Effect = "Allow"
Resource = [
"arn:aws-us-gov:ec2:us-gov-west-1::volume/",
"arn:aws-us-gov:ec2:us-gov-west-1::spot-instances-request/",
"arn:aws-us-gov:ec2:us-gov-west-1::network-interface/",
"arn:aws-us-gov:ec2:us-gov-west-1::launch-template/",
"arn:aws-us-gov:ec2:us-gov-west-1::instance/",
"arn:aws-us-gov:ec2:us-gov-west-1::fleet/",
]
Sid = "AllowScopedResourceCreationTagging"
},
# (10 unchanged elements hidden)
{
Action = "iam:GetInstanceProfile"
Effect = "Allow"
Resource = "arn:aws-us-gov:iam::redacted:instance-profile/"
Sid = "AllowInstanceProfileReadActions"
},
+ {
+ Action = "iam:ListInstanceProfiles"
+ Effect = "Allow"
+ Resource = "
"
+ Sid = "AllowUnscopedInstanceProfileListAction"
},
{
Action = "eks:DescribeCluster"
Effect = "Allow"
Resource = "arn:aws-us-gov:eks:us-gov-west-1:redacted:cluster/itsd-eks-cluster-dev"
Sid = "AllowAPIServerEndpointDiscovery"
},
]
# (1 unchanged attribute hidden)
}
)
tags = {
"managed_by" = "terraform"
"source_code_url" = "redacted"
}
# (7 unchanged attributes hidden)
}

I suspect the issue is related to the policy expansion in combination with deploying in GovCloud partition where the partition identifier and region name are longer.

Region: us-gov-west-1
Partition: aws-us-gov

  • [ x] βœ‹ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

  1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
  2. Re-initialize the project root to pull down modules: terraform init
  3. Re-attempt your terraform plan or apply and check if the issue still persists

Behavior is unchanged after performing these steps.

Versions

  • Module version [Required]: 21.2.0

  • Terraform version:
    Terraform v1.13.2

  • Provider version(s):
    ❯ terraform providers -version
    Terraform v1.13.2
    on darwin_arm64

  • provider registry.terraform.io/hashicorp/aws v6.13.0
  • provider registry.terraform.io/hashicorp/external v2.3.5
  • provider registry.terraform.io/hashicorp/helm v2.17.0
  • provider registry.terraform.io/hashicorp/kubernetes v2.38.0
  • provider registry.terraform.io/hashicorp/local v2.5.3
  • provider registry.terraform.io/hashicorp/null v3.2.4
  • provider registry.terraform.io/hashicorp/random v3.7.2

Reproduction Code [Required]

module "karpenter" {
source = "terraform-aws-modules/eks/aws//modules/karpenter"
version = "~> 21.2"

create = var.karpenter_enabled

cluster_name = data.terraform_remote_state.cluster.outputs.cluster_name
namespace = local.karpenter_namespace

create_pod_identity_association = true

Used to attach additional IAM policies to the Karpenter node IAM role

node_iam_role_additional_policies = {
AmazonSSMManagedInstanceCore = "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

tags = local.tags
}

The variables have the following values:
cluster_name = itsd-eks-cluster-dev
create = true

Steps to reproduce the behavior:

terraform init
terraform plan

Workspaces: no
Cleared local cache: yes
Pulled in new module version: 21.1.5 -> 21.2.0

Expected behavior

The policy updated are applied without receiving the error message.

Actual behavior

An error of "LimitExceeded: Cannot exceed quota for PolicySize: 6144" is generated.

Terminal Output Screenshot(s)

None provided.

Additional context

N/A

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions