New SBC paper on arxiv

[hyunjimoon]

This is the link with the following summary:

Seven case studies
including correct posterior, ignoring data, incorrect correlation, small discrepancies.

Take-aways
i) wouldn't your MCMC estimation (approximated posterior distribution) be sample-based i.e. discrete?
ii) your posterior distribution is only one sample-based approximation of underlying joint distribution, and a member of family of approximated posterior
iii) project your posterior samples to your interest to judge validity of the entire model. model := prior+likelihood+approximator
iv) consider including at least loglik to your list of test quantities

Defined Characters 👶

1. posterior family phi
2. test quantity f
3. sample rank cdf, sample q, sample SBC
4. predictive cdf, tie probability, quantile functions
5. continuous rank cdf, continuous q, continuous SBC
6. ties
7. data-averaged posterior

Stories from interacting characters 👨‍👩‍👧‍👦
Thm1. procedural definition of sample SBC
Thm2. correct posterior and quantile: correct posterior passes continuous SBC
Thm3. characterization of SBC failures
Thm4. continuous SBC implies sample SBC: correct posterior will produce uniform quantile and pass M-sample SBC for every M, f
Thm5. sample SBC implies continuous SBC
Thm6. density ratio: test quantities that flags the difference between wrong posterior and true posterior always exists
Thm7. incomplete use of data: if you ignore some data, you are safe only as long as you use test quantities ignoring that same data
Thm8. monotonic transformations

Appendix B Formal analysis of SBC for a simple model can provide insights on how "test quantities shapes all".