Out of the box install leaves prometheus unable to access pods in other namespaces

[danfinn]

I did a very vanilla install of kube-prometheus today using the instructions in the readme. I then tried to get it to pull metrics from a pod in another namespace by adding a podmonitor but it was saying 0 targets were up. I did some more digging and found this error in my prometheus server logs:

time=2026-03-04T21:47:33.119Z level=ERROR source=reflector.go:204 msg="Failed to watch" component=k8s_client_runtime logger=UnhandledError err="failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"kafka\"" reflector=pkg/mod/k8s.io/client-go@v0.35.0/tools/cache/reflector.go:289 type=*v1.Pod

Seems like the prometheus service account should have the ability to do this out of the box?