feat(Grafana): K8s serviceaccount token as authorization .spec.client.useKubeAuth

[Baarsgaard]

Use the k8s ServiceAccount token projected in at /var/run/secrets/kubernetes.io/serviceaccount/token as authentication to Grafana instances.

Users can configure [auth.jwt] to accept K8s ServiceAccounts and assign them roles depending on the name, namespace, etc in the JWT token and entirely disable the default Grafana admin account.

TODO

• Docs/Examples: https://gist.github.com/Baarsgaard/38217259c5e2f72af28ae8b8792fb54e
  Maybe as a separate PR so it does not show in the docs until released with v5.20.0
• Simplify Grafana configuration by merging: Auth: Support JWT configs tls_client_ca and jwk_set_bearer_token_file grafana#109095

Questions:

• Should the operator support a default config for JWT auth which is used? The variance in what people want might be a bit high and instead just document the options.
• Name of the client option? useKubeAuth and useJWTAuth are both a bit vague
• Should this be merged after Grafana v12.2.0 is released and configured as the new default version?
  Would help a lot for usability as the the full setup can be handled in the Grafana CR

[weisdd]

Should we just omit token validation? - Even though it's relatively easy to implement basic token validation (e.g. using NewProvider + Verifier from github.com/coreos/go-oidc/v3/oidc), it's up to Grafana to do the actual token validation / authorization (like with any other Grafana credentials that the operator uses), the role of the operator here is just to fetch the token from the file and pass it to Grafana when making API requests. - It would also simplify the unit test as you would only need to check that the Authorization header is set to the contents of the file. WDYT?

(We can discuss it further during the meeting)

[Baarsgaard]

Oh yeah, I forgot that TODO comment!
We already discussed this in a weekly meeting and agreed to not validate the token aside from reading the expiration!
I will remove that comment/Change it to the reason why we do not validate the token.

And potentially simplify the test!

[weisdd]

Yeah, I think it was @theSuess who suggested that, I just wanted to surface that conversation here as the code was still in place. We'll wait for further updates then :)

[Baarsgaard]

Looking at it again, since the token lifetime is fully configurable, it's probably a good idea to read the expiration but not validate the token: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#launch-a-pod-using-service-account-token-projection
This allows us to dynamically set the cache timeout of the token if someone decides to shorten the lifetime.

The discussion we had last time was verifying the token signature with the JWKS downloaded from the kubernetes API, which we decided was not worth it?