Proposal to use tags for tracking PowerVS cluster resources

[arshadd-b]

What this PR does / why we need it:
Proposal for adding the tags to PowerVS Cluster resources and performing delete of resources on the bases of tags
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

/area provider/ibmcloud

1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:

Add tags to PowerVS Cluster resources and delete resources based on tags

[k8s-ci-robot]

Hi @arshadd-b. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[netlify]

✅ Deploy Preview for kubernetes-sigs-cluster-api-ibmcloud ready!

Name	Link
🔨 Latest commit	c3f4178
🔍 Latest deploy log	https://app.netlify.com/projects/kubernetes-sigs-cluster-api-ibmcloud/deploys/6895d50d841c090008902d12
😎 Deploy Preview	https://deploy-preview-2364.cluster-api-ibmcloud.sigs.k8s.io
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Amulyam24]

/ok-to-test

[Amulyam24]

/retitle Proposal to use tags for tracking PowerVS cluster resources

[Karthik-K-N]

In the delete flow diagram, I think If is not needed inside the rhombus as its already a decision block
In create flow diagram I think you missed to consider COSInstance

[Karthik-K-N]

Thanks for the PR, Most of the things are good,lets update it bit more to make it better.

[arshadd-b]

In the delete flow diagram, I think If is not needed inside the rhombus as its already a decision block In create flow diagram I think you missed to consider COSInstance

done

[Amulyam24]

Hi @arshadd-b, Thank you for the proposal!

I'm not clear on how user tags will be used apart from the controller tag. Can you please share more details on it?

[arshadd-b]

Hi @arshadd-b, Thank you for the proposal!

I'm not clear on how user tags will be used apart from the controller tag. Can you please share more details on it?

Hi @Amulyam24 , It is the same functionality that we do from UI, adding tags to IBM Cloud resources if user wants to tag resources. It can help to user in resource management.

[Amulyam24]

Hi @Amulyam24 , It is the same functionality that we do from UI, adding tags to IBM Cloud resources if user wants to tag resources. It can help to user in resource management.

Got it, Thanks for the clarification.

A couple of suggestions have been added.

In the create flow diagram, how about we enhance the condition such as cluster.spec.UserTags > 0 , then proceed to attach user provided tags.

[Amulyam24]

Suggested change

	- Currently TransitGateway Connections doesn't support tagging, So we will handle deletion of connections based on VPC.
	- DHCP Server doesn't support tagging, So we will tag DHCP Network and handle deletion based on Network.
	Currently transit gateway connections and DHCP server don't support tagging. We will handle their deletion using the VPC and network tag respectively.

When DHCP server is created, we use its network right, is that taggable?

should we depend on workspace resource tag instead?

cc @Karthik-K-N

[arshadd-b]

@Amulyam24 Actually there is ause case when workspace is already created but DHCP network is newly created,
in that case we want to delete the network only not workspace. So for this case we have to tag the network.
I have already checked for network tagging is supported .

[Karthik-K-N]

I think we can discuss on tag format again here

I see aws using something like sigs.k8s.io/cluster-api-provider-aws/cluster/<cluster-name>: owned
@dharaneeshvrd I remember even hypershift uses tags, whats the format there?

[Karthik-K-N]

What if in future if we want to adopt some pre-created resources, I think this format does not allow it.

[dharaneeshvrd]

kubernetes.io-cluster-<cluster_name>:owned
https://github.com/openshift/hypershift/blob/4f139f8181bb22a26fcffde2067dc8c73705ec5f/cmd/infra/powervs/create.go#L135
This is the format used in hypershift which we got it from IPI at that time.

[Karthik-K-N]

cool thanks, I think having owned helps as later we can add more keywords if required, Then can we stick to this format

sigs.k8s.io/cluster-api-provider-ibmcloud/cluster/<cluster-name>: owned

[Amulyam24]

minor nits, LGTM!

[Prajyot-Parab]

@arshadd-b Please address the review comments and squash the commits.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: arshadd-b / name: arshadd banoo (85da3c1, dcf0feb, 0f4cb6b, c3f4178, 14da794, e6bc0e3)

[Amulyam24]

/lgtm

@arshadd-b, please squash the commits.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Amulyam24, arshadd-b
Once this PR has been reviewed and has the lgtm label, please assign prajyot-parab for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dharaneeshvrd]

How do we handle existing resource which is created by controller in earlier runs which will have controllercreated tag?
E.g.
Due to some reason, resources created during earlier runs did nt get cleaned up properly, so when we process it, it would have the tag but it was nt created as part of current cluster's reconciliation. It would cause collision during destroy flow. We need to find a way to handle this situation too.

[dharaneeshvrd]

How are we going to differentiate the vpc subnet or DHCP network tag for itself and TG connection when only TG connection is created by controller to use them during destroy flow?

[arshadd-b]

vpc subnet and DHCP network supports tagging, So when these resources are newly created and will be tagged.
For TG connection we will check either vpc subnet or DHCP network since TG connnection doesn't support tagging.

[dharaneeshvrd]

Yes, my question is what are the labels you would use when TG connection is created by controller when vpc subnet and DHCP network also created by controller? Dont you need to add two label to the vpc subnet? one is to denote its own itself and another one for vpc TG connection. In that case dont you need to use a different tag format? I feel that also need to be documented properly here.

[Karthik-K-N]

I understand it like this

1. Transit Gateway supports tagging but its connections does not support tagging

Deletion flow

Case 1: All TG and its connections created by controller

If TG has tag attached then it means TG and its connections are created by controller then we can directly delete all 3 resources(TG, PowerVS Connection, VPC Connections)

Case 2: Pre-existing TG but new connections

1. TG tag does not match the cluster in deletion's tag means we are not suppose to delete it
2. Now to delete PowerVS Connection, Get the network name of connection and check whether the network has the cluster in deletion's tag, If present delete
3. Same for VPC.

Is my understanding correct or do you think is there any gap?

[arshadd-b]

There is one more scenario

case 3: What if only connections are created but TG and networks(vpc subnet and DHCP network) pre-existing.
In this case we have to delete only connections, to handle this case we need to add one tag to TG with below format
sigs.k8s.io/cluster-api-provider-ibmcloud/cluster/<cluster-name>/TG: connection1, connection2

Then before deleting we will check for this tag,
will be adding this in proposal

[arshadd-b]

How do we handle existing resource which is created by controller in earlier runs which will have controllercreated tag? E.g. Due to some reason, resources created during earlier runs did nt get cleaned up properly, so when we process it, it would have the tag but it was nt created as part of current cluster's reconciliation. It would cause collision during destroy flow. We need to find a way to handle this situation too.

I was thinking since we are not removing controllercreated field completely
So first we will check the tag is there or not if not tagged , then we will check controllercreated field then delete the resource

[dharaneeshvrd]

The final goal would be to remove that totally right, So depending on it won't be the right thing to do. We need to have clear plan on how to handle this if we are implementing tag based way of identifying the resources created by controller.

[Karthik-K-N]

How do we handle existing resource which is created by controller in earlier runs which will have controllercreated tag? E.g. Due to some reason, resources created during earlier runs did nt get cleaned up properly, so when we process it, it would have the tag but it was nt created as part of current cluster's reconciliation. It would cause collision during destroy flow. We need to find a way to handle this situation too.

Good point, I think as a transition plan I think we should check whether the resource has controllerCreated is set to true, In that case we should go ahead and create the tag. In this case we can completely depend on the tag to delete.

[arshadd-b]

How do we handle existing resource which is created by controller in earlier runs which will have controllercreated tag? E.g. Due to some reason, resources created during earlier runs did nt get cleaned up properly, so when we process it, it would have the tag but it was nt created as part of current cluster's reconciliation. It would cause collision during destroy flow. We need to find a way to handle this situation too.

Good point, I think as a transition plan I think we should check whether the resource has controllerCreated is set to true, In that case we should go ahead and create the tag. In this case we can completely depend on the tag to delete.

yeah makes sense, will add this trasition plan in proposal

[dharaneeshvrd]

Good point, I think as a transition plan I think we should check whether the resource has controllerCreated is set to true, In that case we should go ahead and create the tag. In this case we can completely depend on the tag to delete.

With this design it feels like we would need the controllercreated status as a dependency for attaching tags. But I feel we should have some idea on how can we move away from controllercreated status in future. Because after implementing this, if we always dependant on controllercreated status to handle certain cases, then what's the point of implementing tags?

[Karthik-K-N]

Good point, I think as a transition plan I think we should check whether the resource has controllerCreated is set to true, In that case we should go ahead and create the tag. In this case we can completely depend on the tag to delete.

With this design it feels like we would need the controllercreated status as a dependency for attaching tags. But I feel we should have some idea on how can we move away from controllercreated status in future. Because after implementing this, if we always dependant on controllercreated status to handle certain cases, then what's the point of implementing tags?

This is just a transition layer, The ultimate goal is to remove the controllerCreated field.

When we have the tagging support

1. For already existing cluster if it has controllerCreated we will create a tag
2. For new cluster we always create a tag.

This would help us in deletion flow to soley depend on the tag rather than the controllerCreated field and eventually we will remove the field in the later releases.