Allow reloading CA cert pool in tlsConfig on certificate renewal and reload handled by certwatcher

[akankshapanse]

Currently certwatcher package in controller-runtime allows watching server level TLS certificate and also provides APIs such as GetCertificate(), always returns latest current server certificate with watcher reloading/re-reading cert on renewal.
tls.Config{} provides option GetCertificate to be set to function, that gets invoked when client requests certificate from given server.
Setting tls.Config.GetCertificate to certwatcher.GetCertificate ensures tlsConfig always reads and provides current latest certificate while performing TLS communcation from given server.

There are 2 issues observed here:

1. There is no GetClientCertificate() API available in certwatcher, which can be assigned to tls.Config.GetClientCertificate to fetch and provide client certificate whenever any server requests certificate from given client.
2. There is no API available in tls.Config{} and certwatcher to set/reset CA cert pool in case CA cert has renewed as well. Similar to options GetCertificate(), GetClientCertificate(), we need additional options such as GetRootCAs(), GetClientCAs() etc. in both tls.Config{} and certwatcher package, so that we can re-read/reload CA cert after their renewal for renewal of given child certificate.
   Currently, if CA cert gets renewed, there is no way in tls.Config to reload them to be used for TLS communication between any server or client, even with certwatcher.
   Here are certain issues already reported in cert-manager and not resolved in code yet.
   CA cert in Secret not updated when self-signed CA itself gets renewed. cert-manager/cert-manager#5851
   Certmgr allows creating certificates expiring after ca expiration. cert-manager/cert-manager#5864

Please add/correct if anything missed.

[sbueringer]

If I got it right sounds like 2 would require some changes in the Go SDK?

[akankshapanse]

If I got it right sounds like 2 would require some changes in the Go SDK?

for certwatcher, changes would be needed in controller-runtime package. For tls.Config, changes would be needed in go TLS library