Report preemption costs on batch details page #15220
Description
The user request is for a rolling "cost of preempted attempts" (in both dollars and time) on the batch details dashboard alongside the overall total cost.
We should be careful not to increase page load time - or database load. We could potentially track this as we go along (like we do with the terminal state counter metrics) or make this an "on demand" stat that we calculate separately, if requested, after the page has already loaded.
As a side note: I'm a little nervous about reporting this without also reporting a "saved by using preemptible machines" dollars stat as well. I don't want users to walk away with the impression that they lost money by using preemptible machines (unless they truly did, but the bar for that is a lot higher)