Adding grafana dashboards on the fly

[hakkyou]

Hey!
Can I add persistent grafana dashboards on-the-fly without using helm chart?
I mean, is it good idea to add grafana dashboards dynamically using labels on configmaps instead of recompiling manifests (or adding configmaps to compiled manifest manually).
I know that I can add grafana dashboards in configmaps when using helm chart (grafana_dashboard: 1), but helm is imperative, not declarative, and we are trying to be as declarative as possible in our k8s infra.

[paulfantom]

AFAIK helm chart uses a special sidecar to allow this. In the case of kube-prometheus we don't have that sidecar so you need to regenerate jsonnet code if you want to add a new dashboard. This way we can ensure deletion works as there won't be any stale objects left when doing kubectl delete.