Creating the vpc-cni addon after creating a new EKS cluster fails due to PolicyEndpoint CRD conflict.

[tommyf-smilecdr]

Problem Summary

Creating the vpc-cni addon after creating a new EKS cluster fails due to PolicyEndpoint CRD conflict.

Problem Details

If you create a new EKS cluster after November 13th 2025 without the vpc-cni addon, attempting to add it after the fact, will result in the following error:

code: ConfigurationConflict
message: Conflicts found when trying to apply. Will not continue due to resolve conflicts mode. Conflicts: CustomResourceDefinition.apiextensions.k8s.io policyendpoints.networking.k8s.aws - .spec.versions

Steps to reproduce

1. Create EKS Cluster without the vpc-cni addon. The EKS version does not seem to matter. I have only tried with 1.34 and 1.33.
   This can be done in a number of ways with the same result:
   • Using eksctl or the aws cli.
   • Using the AWS console using EKS Auto mode (This will not install the vpc-cni addon)
   • Using the AWS console WITHOUT using EKS Auto mode. Ensure to de-select the vpc-cni addon on the addons page.
   • Using the Terraform eks resource, without adding any cluster_addon items.
2. Create the vpc-cni addon. As above, this can be done in a number of ways with the same result. I will only demonstrate the aws cli method that follows the official docs here:
   • Create the vpc-cni addon.
     aws eks create-addon --cluster-name my-cluster --addon-name vpc-cni --addon-version v1.20.3-eksbuild.1
   • Either observe the addon status in the AWS Console or check using the aws cli...
     aws eks describe-addon --cluster-name my-cluster --addon-name vpc-cni
   • Observe that the addon has the CREATE_FAILED status with the following:

   {
       "addon": {
           "addonName": "vpc-cni",
           "clusterName": "my-cluster",
           "status": "CREATE_FAILED",
           "addonVersion": "v1.20.4-eksbuild.1",
           "health": {
               "issues": [
                   {
                       "code": "ConfigurationConflict",
                       "message": "Conflicts found when trying to apply. Will not continue due to resolve conflicts mode. Conflicts:\nCustomResourceDefinition.apiextensions.k8s.io policyendpoints.networking.k8s.aws - .spec.versions"
                   }
               ]
           },
           ...
       }
   }
   

Affected Versions

• In my existing workflow I was using the terraform method mentioned above.
• I had deployed some time on November 12th/13th without issue (i.e. Create cluster without the addon, and then create the addon separately).
• Attempting this deployment on November 14th failed as desribed above.
• I discovered that a similar issue (PolicyEndpoint CRD deletion on addon upgrade can lead to indefinite network interruption in rare circumstances #199 ) was raised on Nov 13th, I do wonder if they are somehow related.
• I see that the domainName field was added to the CRD in the as-of-yet unreleased release-v1.1 branch early in the morning on November 14th 2025 ( 7a70271 )

Workaround

It is easy to work around this issue if you set the conflict resolution method to Override, but this is not an intuitive thing to do when adding a NEW addon, and is not included in the official docs for doing so. It also skirts around the core problem of the mismatched PolicyEndpoint CRDs in use.

Suspected Cause of Problem

Upon further investigation, it appears that when creating a new EKS cluster, it uses the version of the CRD that is included in the upcoming 1.1 release of the network policy controller. This new version of the CRD contains a new domainName field inside the egress and ingress rule schemas, but the version of the API remains at v1alpha1.

I don't know the mechanisms used by the AWS Control Plane when creating an EKS cluster, but based on the evidence, it seems that it likely uses the Helm Chart in this repository to deploy the policy controller?

If this is the case, then it is worth noting that v1.1 of the Helm Chart release installs the v1.1 CRD, but also installs the existing v1.0.x of the CRD watcher component. The result of this is that if you manually remove the PolicyEndpoint CRD, when it gets reconciled, it is replaced by the v1.0.x of the CRD and NOT the v1.1 of the CRD that gets installed with a new EKS cluster.

It seems to me that changes in the schema for a CRD should be accompanied by a bump in the schema version to avoid such conflicts. I do understand that this is an alpha schema version, which means that breaking changes may occur at any time. This would be fine if it were not for the fact that this, potentially breaking, schema is included in all production EKS clusters that may rely on stable schemas.

Summary

• Release 1.1 of the network policy controller should not be released until the CRD created at deploy time match the CRD created by the watcher.
• EKS clusters should not be created using this unreleased version of the network policy controller.
• Production EKS clusters should not be created with core components that rely on CRDs with version v1alpha1 as this can lead to unpredictable behaviour as we see here.
  • If they do, then there needs to be guarantees that the v1alpha1 CRD never changes.
  • Structural changes to any CRD should result in a new version (e.g. v1alpha2)

[sienna011022]

I’m also encountering the same issue when installing the vpc-cni add-on AFTER creating an EKS cluster.

Environment & details:

• EKS cluster version: 1.34 (newly created)
• Existing CRD: policyendpoints.networking.k8s.aws
• controller-gen.kubebuilder.io/version: v0.11.3
• API version: v1alpha1
• Status: Established
• Created at: 2025-11-14T07:55:49Z

When I try to install vpc-cni

Conflicts found when trying to apply. …
CustomResourceDefinition.apiextensions.k8s.io policyendpoints.networking.k8s.aws – .spec.versions

[oliviassss]

@tommyf-smilecdr Hi, thanks for the reporting.
We have noticed this issue and are working on a fix. The VPC CNI addon shouldn't manage the CRD moving forward, the conflict will be resolved after we roll out the fix from addon side.

@sienna011022, to resolve the addon conflict on CRD management, please use the --resolve-conflicts OVERWRITE as a short-term mitigation until the fix rolled out.

[nApucco]

@oliviassss

So if you're creating vpc-cni addon after cluster creation, please use the --resolve-conflicts PRESERVE as a short-term mitigation until the fix rolled out.

Don't you mean --resolve-conflicts OVERWRITE?
That was the only way I was able to successfully install the vpc-cni addon after cluster creation.
(with PRESERVE the installation still fails, since a conflict is detected)

[tommyf-smilecdr]

Don't you mean --resolve-conflicts OVERWRITE?
That was the only way I was able to successfully install the vpc-cni addon after cluster creation.
(with PRESERVE the installation still fails, since a conflict is detected)

I agree with @nApucco on this. It fails in this scenario - I forgot to add that to the initial issue.

In addition, when adding the addon, there is no option (in the AWS Console at least) to use the PRESERVE mode. You must use OVERWRITE.

An additional point of note is that the Policy reconciler component of the network policy controller will use the OLD version of the CRD if it gets deleted. The only time the new version of the CRD will be created is during the initial deployment of a new cluster.

[oliviassss]

@tommyf-smilecdr looks like the Addon option may have a gap in cli and console, this cmd works in my setup

% aws eks create-addon --cluster-name dev-npc-sonyingy --addon-name vpc-cni --resolve-conflict PRESERVE

Per the doc it should support PRESERVE as a value for --resolve-conflicts in creation workflow as well: https://docs.aws.amazon.com/eks/latest/APIReference/API_CreateAddon.html#AmazonEKS-CreateAddon-request-resolveConflicts

In additional point of note is that the Policy reconciler component of the network policy controller will use the OLD version of the CRD if it gets deleted. The only time the new version of the CRD will be created is during the initial deployment of a new cluster.

We identified this flaw in NPC as well is rolling out a version with fix. However, existing customers using NetworkPolicy and PolicyEndpoint should not be affect. The new PE CRD with domainName is related to a feature we are to release. New cluster, or instance refresh on existing cluster should have the new PE CRD installed.

@nApucco, I have updated my previous comment, thanks for pointing out.

[oliviassss]

UPDATE:

• As of 11/19/2025, the fix from EKS addon has been rolled out to all commercial regions. Customers shall not have any conflict on PE CRD when upgrading vpc-cni addon.

• As of 11/19/2025, EKS is re-releasing the affected addon versions VPC CNI v1.20.4-eksbuild.2 and v1.20.4-eksbuild.3. ECD to deploy for all commercial regions is 11/21/2025.