Skip to content

✨ allow preserving addons on delete #5520

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 12 commits into from
Sep 4, 2025
Merged

✨ allow preserving addons on delete #5520

merged 12 commits into from
Sep 4, 2025

Conversation

alam0rt
Copy link
Contributor

@alam0rt alam0rt commented Jun 4, 2025 •
edited by richardcase
Loading

What type of PR is this?

feature

What this PR does / why we need it:

We are looking at migrating away from the managed addons (specifically kube-proxy) and wanting to use our own self-managed.

The issue is that both kube-system/daemonset/kube-proxys can't exist at the same time for obvious reason.

Removing the addon from .spec.addons will of course, delete the addon.

Deploying over the addon works, but again, removing the addon will remove the now self-managed addon.

The only way (without juggling multiple daemonsets, using taints/tolerations to run both at the same time, etc) for a clean migration is to (please let me know if a good way comes to mind):

  • Deploy over the managed addon
  • Remove the addon while preserving the resources on the cluster

We could potentially stop CAPA reconciliation and remove the addon manually, but kind of want to avoid that.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Checklist:

  • squashed commits
  • includes documentation
  • includes emoji in title
  • adds unit tests
  • adds or updates e2e tests

Release note:

Allow preserving EKS addons.

@k8s-ci-robot k8s-ci-robot added do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jun 4, 2025
@k8s-ci-robot k8s-ci-robot requested review from Ankitasw and cnmcavoy June 4, 2025 06:53
@k8s-ci-robot
Copy link
Contributor

Hi @alam0rt. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 4, 2025
@alam0rt
Copy link
Contributor Author

alam0rt commented Jun 4, 2025

/kind feature

@k8s-ci-robot k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Jun 4, 2025
@alam0rt
Copy link
Contributor Author

alam0rt commented Jun 4, 2025

Speaking with my team, would it be better if the preserve field was in sigs.k8s.io/cluster-api-provider-aws/v2/controlplane/eks/api/v1beta1.Addon?

image

@alam0rt
Copy link
Contributor Author

alam0rt commented Jun 5, 2025

Speaking with my team, would it be better if the preserve field was in sigs.k8s.io/cluster-api-provider-aws/v2/controlplane/eks/api/v1beta1.Addon?
image

I went and did it. Seems cleaner.

@richardcase
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 8, 2025
@richardcase
Copy link
Member

@alam0rt - thanks for this. Just so i understand this correct. If you want to delete and use your own version of the addon, is this the process:

  • Define the kube proxy addon with preserveOnDelete=true in AWSManagedControlPlane
  • Delete the addon from AWSManagedControlPlane

@richardcase
Copy link
Member

Also, guessing this is only for the use case where bootstrapSelfManagedAddons wasn't specified at cluster creation time?

@alam0rt
Copy link
Contributor Author

alam0rt commented Jun 10, 2025

@alam0rt - thanks for this. Just so i understand this correct. If you want to delete and use your own version of the addon, is this the process:

  • Define the kube proxy addon with preserveOnDelete=true in AWSManagedControlPlane

  • Delete the addon from AWSManagedControlPlane

That's correct!

@richardcase
Copy link
Member

/test ?

@k8s-ci-robot
Copy link
Contributor

@richardcase: The following commands are available to trigger required jobs:

/test pull-cluster-api-provider-aws-build

/test pull-cluster-api-provider-aws-build-docker

/test pull-cluster-api-provider-aws-test

/test pull-cluster-api-provider-aws-verify

The following commands are available to trigger optional jobs:

/test pull-cluster-api-provider-aws-apidiff-main

/test pull-cluster-api-provider-aws-e2e

/test pull-cluster-api-provider-aws-e2e-blocking

/test pull-cluster-api-provider-aws-e2e-clusterclass

/test pull-cluster-api-provider-aws-e2e-conformance

/test pull-cluster-api-provider-aws-e2e-conformance-with-ci-artifacts

/test pull-cluster-api-provider-aws-e2e-eks

/test pull-cluster-api-provider-aws-e2e-eks-gc

/test pull-cluster-api-provider-aws-e2e-eks-testing

Use /test all to run the following jobs that were automatically triggered:

pull-cluster-api-provider-aws-apidiff-main

pull-cluster-api-provider-aws-build

pull-cluster-api-provider-aws-build-docker

pull-cluster-api-provider-aws-test

pull-cluster-api-provider-aws-verify

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@richardcase
Copy link
Member

/test pull-cluster-api-provider-aws-e2e-eks

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jun 12, 2025
@richardcase
Copy link
Member

/approve

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 12, 2025
@kubernetes-sigs kubernetes-sigs deleted a comment from k8s-ci-robot Jun 12, 2025
@MinhNguyen-at MinhNguyen-at mentioned this pull request Jun 13, 2025
Open
@Ankitasw
Copy link
Member

/test pull-cluster-api-provider-aws-e2e-blocking

@Ankitasw
Copy link
Member

Ankitasw commented Sep 1, 2025

/retest

@Ankitasw
Copy link
Member

Ankitasw commented Sep 1, 2025

/lgtm

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: richardcase

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 4, 2025
@alam0rt alam0rt force-pushed the sam.lockart/allow-preserving-addons branch from 694281b to b33add2 Compare September 4, 2025 03:53
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. do-not-merge/contains-merge-commits and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 4, 2025
@alam0rt alam0rt force-pushed the sam.lockart/allow-preserving-addons branch from 8718bc2 to b33add2 Compare September 4, 2025 04:00
@alam0rt alam0rt force-pushed the sam.lockart/allow-preserving-addons branch from b33add2 to c109cab Compare September 4, 2025 04:02
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 4, 2025
@alam0rt
Copy link
Contributor Author

alam0rt commented Sep 4, 2025

/retest

1 similar comment
@Ankitasw
Copy link
Member

Ankitasw commented Sep 4, 2025

/retest

@Ankitasw
Copy link
Member

Ankitasw commented Sep 4, 2025

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 4, 2025
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 55cec9f4995685c98bfcd53c9084c54c20ccae2d

@k8s-ci-robot k8s-ci-robot merged commit e906ef4 into kubernetes-sigs:main Sep 4, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Ankitasw Ankitasw Awaiting requested review from Ankitasw

@cnmcavoy cnmcavoy Awaiting requested review from cnmcavoy

Assignees

@Ankitasw Ankitasw

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@alam0rt @k8s-ci-robot @richardcase @Ankitasw