test deleting and recreating konk and konkservice

[kd7lxl]

Secrets created by cert-manager are not owned by the helm chart, so they don't get cleaned up by the helm operator after deleting a Konk or KonkService. After deleting and recreating a Konk, it will have a new CA, but secrets created by cert-manager signed by the old CA will still exist. cert-manager won't attempt to recreate these until they reach their renewal period. This can result in a bunch of x509/auth errors after trying to delete and recreate a chart that uses konk, because the kubeconfig-cert will be stale.

One solution to this is to enable the global cert-manager option --enable-certificate-owner-ref=true. Note: we already do this in the tests:

konk/Makefile

Line 46 in af0da85

--set extraArgs[0]="--enable-certificate-owner-ref=true""

Another option would be to not put any ownerReferences on the konk-ca secret, so that it persists across delete/recreate events.

konk/helm-charts/konk/scripts/provision.sh

Lines 98 to 105 in af0da85

	for name in $FULLNAME-apiserver-cert $FULLNAME-etcd-cert $FULLNAME-ca $FULLNAME-etcd-ca $FULLNAME-kubeconfig
	do
	kubectl patch -n $NAMESPACE secret $name -p '{"metadata":{"ownerReferences":[{"apiVersion":"apps/v1", "kind":"Deployment", "name":"'${FULLNAME}'", "uid":"'${DEPLOYMENT_UID}'"}]}}'
	done
	if [ "$SCOPE" = "cluster" ]
	then
	kubectl patch -n $CERT_MANAGER_NAMESPACE secret $FULLNAME-ca -p '{"metadata":{"ownerReferences":[{"apiVersion":"apps/v1", "kind":"Deployment", "name":"'${FULLNAME}'", "namespace":"'${NAMESPACE}'", "uid":"'${DEPLOYMENT_UID}'"}]}}'
	fi

https://infoblox.atlassian.net/browse/ATLAS-9946