The Prometheus service breaks when using a custom docker volume driver. Basically, the container does not have permission to write to the /prometheus folder. This is because root permission is needed to write to that folder when using a volume driver (specifically rancher-nfs in my case). The default user for the Prometheus container is nobody, hence the write permission error.