Manage Grafana Service Accounts from the Grafana CR

[ndk]

#1469 · feat: declarative Grafana Service Account management

Design proposal: #003

---

Why

The Grafana Operator lets you manage Grafana through Kubernetes CRs, but service accounts were still a manual step (GUI or HTTP API). This PR lets you declare SAs in the Grafana CR so the operator can:

• Manage service accounts in Grafana
• Generate API tokens
• Store each token in a Kubernetes Secret
• Clean everything up when the CR changes or is removed

What's inside

• New GrafanaServiceAccount CR.
• Full management flow for service accounts, tokens, and secrets.
• Operator records managed items in status.serviceAccounts and exposes conditions.
• Chainsaw e2e: tests/e2e/grafanaserviceaccount.

Design notes

• orgId defaults to the default organization

Out of scope (for now)

• Multi-org support. All calls target the default Grafana org.
• Cross-namespace Secret writes.
• Automatic token rotation (expires).
• Permissions (it's an Enterprise/Cloud-only feature).

Known limitations

• Token secrets are always written to the same namespace as the GrafanaServiceAccount.

TODO

• Unit tests for SA reconciler logic.
• Token rotation.
• User docs and sample manifests.
• Feature toggle.
• Metrics.
• Validation.
• Watch token Secrets (Owns).
• ServiceAccounts: return 404 when deleting a non‑existent service account grafana#106618
• fix: allow sending isDisabled=false in UpdateServiceAccountForm grafana-openapi-client-go#115
• chore: update schema to 12.1.0 grafana-openapi-client-go#120

CR example

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaServiceAccount
metadata:
  name: mysa
spec:
  name: my-service-account
  role: Admin
  instanceName: grafana
  isDisabled: false
  tokens:
    - name: my-token
      secretName: ddd
      expires: 2029-12-31T14:00:00+02:00

[Baarsgaard]

I will leave it this for now and review the remaining functions when I have more time to familiarize myself with the service accounts api :)

[theSuess]

Overall code & flow looks good to me! Left some minor points of improvement but nothing major. Good work @ndk!

[Baarsgaard]

If it's not too much to ask, would you include an example of using the new CR under examples/serviceaccounts/?

[theSuess]

LGTM! Let's see if @Baarsgaard has any last comments - other than that, I'm happy to merge this

[Baarsgaard]

While testing this I found a minor oversight, which might be a regression from an earlier version:
The operator currently fails to take ownership if a Service account with the same name already exists in Grafana:

status:
  conditions:
  - lastTransitionTime: "2025-08-30T21:23:29Z"
    message: |-
      ServiceAccount failed to be applied for 1 out of 1 instances. Errors:
      - grafana: upserting service account: creating service account: [POST /serviceaccounts][400] createServiceAccountBadRequest {"message":"service account already exists"}
    observedGeneration: 1
    reason: ApplyFailed
    status: "False"
    type: ServiceAccountSynchronized

Sidenote @theSuess, does it ever make sense to create a GrafanaServiceAccount with an empty token list?
Currently, if the token list is empty, the operator will delete any manually created tokens as outlined in the Proposal, but is there any use for serviceaccounts that does not contain tokens?
If not, should there be a MinItems validation on the .spec.tokens field?

[Baarsgaard]

Everything else I tested worked flawlessly! Really nice work @ndk :D

[Baarsgaard]

Last note:
Most other CRs that have a .spec.title or .spec.name have them as optional and derive the name from the .metadata.name if omitted.
Should the same be done here with the difference being that .spec.name remains immutable and cannot be updated after creation?

[ndk]

While testing this I found a minor oversight, which might be a regression from an earlier version: The operator currently fails to take ownership if a Service account with the same name already exists in Grafana:

status:
  conditions:
  - lastTransitionTime: "2025-08-30T21:23:29Z"
    message: |-
      ServiceAccount failed to be applied for 1 out of 1 instances. Errors:
      - grafana: upserting service account: creating service account: [POST /serviceaccounts][400] createServiceAccountBadRequest {"message":"service account already exists"}
    observedGeneration: 1
    reason: ApplyFailed
    status: "False"
    type: ServiceAccountSynchronized

Sidenote @theSuess, does it ever make sense to create a GrafanaServiceAccount with an empty token list? Currently, if the token list is empty, the operator will delete any manually created tokens as outlined in the Proposal, but is there any use for serviceaccounts that does not contain tokens? If not, should there be a MinItems validation on the .spec.tokens field?

On the existing service account case. There's no perfect solution. If we take ownership, a simple typo in the name could end up wiping tokens and breaking existing workflows. I'd rather surface the conflict so the user decides what to do. Maybe a flag to control this behavior (takeOwnership: true) could work.

As for empty token lists. I'd stick to mirroring Grafana’s API. If someone wants a service account without tokens, why block it? Adding minItems doesn't really help UX, it just limits flexibility.

[Baarsgaard]

On the existing service account case. There's no perfect solution. If we take ownership, a simple typo in the name could end up wiping tokens and breaking existing workflows. I'd rather surface the conflict so the user decides what to do. Maybe a flag to control this behavior (takeOwnership: true) could work.

We discussed this in the weekly a while back, if I recall correctly, the reasoning is twofold:

1. The same risk/behaviour is present for all the other CRs and this behaving differently would be weird.
2. In a disaster recovery scenario where external Grafanas are in play, the user will have to manually clean up old service accounts if the cluster state is lost, either by patching all CRs with the suggested field or delete the account manually by hand or script.

I checked the revised proposal and it's included under Scopes and limitations

As for empty token lists. I'd stick to mirroring Grafana’s API. If someone wants a service account without tokens, why block it? Adding minItems doesn't really help UX, it just limits flexibility.

Depends on which behaviour is considered more confusing?
If service accounts have no use without a token, then creating valid service accounts without tokens might lead users to think they have to create a token manually, which is then automatically removed shortly after on the next reconcile.

Where as the opposite would be that they cannot apply the resource without at least one token name defined.