Avoid target group reconcile due to externally registered targets.

[dahamaws]

Is your feature request related to a problem?
I am using AWS controller for EKS to create NLB, Target group and listeners. Also using the EKS Automode built in LB controller to register targets using TargetRoleBinding.

The problem is once the targets are registered to the target group, ACK detect the TargetGroup resource state has changed and it automatically deregister the targets. Target groups are highly dynamic and its is not particle to hard code them in TargetGroup objects. Can we have a feature to avoid deleting externally created targets.

TargetGroup object

apiVersion: elbv2.services.k8s.aws/v1alpha1
kind: TargetGroup
metadata:
  name: app-target-group
  namespace: default
  annotations:
    services.k8s.aws/adoption-policy: "adopt"
spec:
  name: app-tg
  port: 80
  protocol: TCP
  targetType: ip
  vpcID: vpc-xxxxxxee0327141cc   # Replace with your VPC ID
  healthCheckProtocol: TCP
  healthCheckPort: "80"
  healthCheckIntervalSeconds: 30
  healthyThresholdCount: 3
  unhealthyThresholdCount: 3

Describe the solution you'd like
ELB2 controller does not mark externally registered target as a state change.

Describe alternatives you've considered

The workaround I have is removing TargetGroup object from here.

      - name: RECONCILE_RESOURCES
          value: Listener,LoadBalancer,Rule, TargetGroup

[github-actions]

Hello @dahamaws 👋 Thank you for opening an issue in ACK! A maintainer will triage this issue soon.

We encourage community contributions, so if you're interested in tackling this yourself or suggesting a solution, please check out our Contribution and Code of Conduct guidelines.

You can find more information about ACK on our website.

[k8sdevlk]

Controller should allow to reconcile all other features and have a configuration option to ignore target registration or de-registration.

It's confirmed that if targets are registered via EKS Auto Mode using TargetGroupBinding, elbv2 controller would remove the targets during reconciliation period (controller restarts or 10hours) - it should ignore targets registered by other means ie TargetGroupBinding or manual instead of removing.

Would love to have this fix in - special thanks!

[k8sdevlk]

Removing TargetGroup from

     - name: RECONCILE_RESOURCES
          value: Listener,LoadBalancer,Rule,TargetGroup

this also prevents controller from creating new targetgroup - essentially disable targetgroup creation...

[rushmash91]

Thank you for raising this issue.

Once the resources are created, add the "ReadOnly" annotation to the TargetGroup resource in Kubernetes. This can be done by updating the TargetGroup object with: https://aws-controllers-k8s.github.io/community/docs/user-docs/features/#readonlyresources

metadata:
  annotations:
    services.k8s.aws/read-only: "true"

With this annotation, the ACK controller will stop reconciling changes to the targets, while still allowing updates to other aspects of the TargetGroup.

To implement:

1. Create/Adopt the resources using ACK without the ReadOnly annotation.
2. Once created, update the TargetGroup resource to add the ReadOnly annotation.

[k8sdevlk]

Thank you @rushmash91 - Will try out the annotations as suggested.

Special thanks,

K

[rushmash91]

Feel free to reopen the issue if you continue to encounter an issue or have further issues 🙂
/close

[ack-prow]

@rushmash91: Closing this issue.

In response to this:

Feel free to reopen the issue if you continue to encounter an issue or have further issues 🙂
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.