Marketplace: Uninstalling the marketplace extension does not remove resources

[mikkelhegn]

I uninstalled the SpinKube extension from the Azure portal. After completing the uninstall, a SpinApp was still running, and the CRDs were still installed in the cluster. Not sure what the intention is here, but it would be great to:

1. Block uninstall if resources are running
2. Remove everything the extension installed in the first place

[Mossaka]

The expectation here is that uninstalling the SpinKube extension should deletes all the CRDs.

"Helm lacks support for removing Custom Resource Definitions (CRDs).... One option is to craft custom Helm hooks to manage CRD deletion within the Helm release lifecycle. These hooks enable the execution of tailored cleanup procedures, such as CRD removal, during uninstallation."

[macolso]

This seems like a higher priority issue to resolve before releasing the marketplace offering. @vdice or @Mossaka - would either of you have time to take a look into the custom Helm hook for removing CRDs?

[vdice]

1. Block uninstall if resources are running

Two questions:

1. @mikkelhegn which resources do you mean? One or more Spin Apps I presume? I can definitely see wanting to block uninstall or alert the user Spin Apps are running on the cluster, as users will no longer be able to deploy/update (and delete?) these if we uninstall SpinKube and CRDs
2. @Mossaka is there a mechanism in the marketplace to do such a thing as above? (i.e. block attempted uninstallation of an extension)
   a. If there isn't a way to do so, are we comfortable allowing uninstallation of the extension when Spin Apps are still running on the cluster?

[vdice]

Pulling this in to look at CRD removal via helm hook, even if thoughts on #26 (comment) are still pending.

[mikkelhegn]

1. Block uninstall if resources are running

Two questions:

1. @mikkelhegn which resources do you mean?  One or more Spin Apps I presume?  I can definitely see wanting to block uninstall or alert the user Spin Apps are running on the cluster, as users will no longer be able to deploy/update (and delete?) these if we uninstall SpinKube and CRDs

I think some kind of message, saying "You're still using this stuff, uninstall blocked" would be good. And then the whole deleting everything comes after. Does that make sense? I have no idea if it's possible, but I'd expect a "solution" to deliver this type of experience.

[Mossaka]

Imo, the expectation should be that uninstalling the extension removes the CRDs from the cluster, but the running Spin application should not be affected.

[mikkelhegn]

but the running Spin application should not be affected

Do you mean that the Deployment should not be affected? I'm assume the SpinApp resource will not be able to exist in the cluster without the CRD.

[vdice]

As @mikkelhegn mentions and confirmed by my testing: Deleting the CRD deletes any and all associated CRs. So, all Spin Apps would be deleted from the cluster if/when the SpinApp CRD is deleted. (Same with SpinAppExecutor CRD/CRs.) Since the underlying Spin App Pod(s) are associated with its corresponding SpinApp resource, these are also deleted.

Therefore, I definitely think we want some way to block uninstall if custom resources are still present. The only idea I can think of from the Helm-side of things is to look into a 'pre-delete' webhook where we see if any SpinApp (or SpinAppExecutor) custom resources exist; if so, fail the hook, which I believe should fail the helm delete command, which I think in turn would fail the extension uninstall in the marketplace -- though I have no idea if the failure/error would be clear or discoverable to the user. I'll look into this today to see about its feasibility.

The better user experience would be to present the error at the extension level, requiring some component in the Azure extension management system to detect the existence of these custom resources and block uninstall... will also research.

[vdice]

Added a PR for the helm hooks in #29

Need to research Azure Marketplace options next. edit: Currently there are no mechanisms in place to prevent uninstallation of an extension in the Marketplace

[vdice]

Closed by #29