Configure startupTaints for karpenter NodePools (#1506)

* Configure startupTaints for karpenter NodePools

* Improve changelog

Author: fiunchinho

CHANGELOG.md

@@ -14,6 +14,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Switch to HelmReleases to install `karpenter` and `karpenter-crossplane-resources` charts.
 - Bump flux `HelmReleases` api version to v2.
 - Reduce heartbeat timeout for ASG lifecycle hooks to from 30 minutes to 3 minutes since aws-node-termination-handler-app (NTH) can now send heartbeats
+- Configure the following `startupTaints` to help `karpenter` ignore pending `Pods` due to these taints that will be removed after the node starts, avoiding unnecessary instance provisioning:
+  - `node.cluster.x-k8s.io/uninitialized:NoSchedule`
+  - `node.cilium.io/agent-not-ready:NoSchedule`
+  - `ebs.csi.aws.com/agent-not-ready:NoExecute`
 
 ### Removed
 

helm/cluster-aws/templates/_karpenter_machine_pools.tpl

@@ -136,9 +136,15 @@ spec:
           - linux
         {{- end }}
         startupTaints:
-        - effect: NoExecute
+        - effect: NoSchedule
+          key: node.cluster.x-k8s.io/uninitialized
+          value: "true"
+        - effect: NoSchedule
           key: node.cilium.io/agent-not-ready
           value: "true"
+        - effect: NoExecute
+          key: ebs.csi.aws.com/agent-not-ready
+          value: "true"
         {{- with $value.customNodeTaints }}
         taints:
         {{- range . }}