Add posterior SBC support

[currocam]

Description

This PR adds support for posterior SBC by adding an optional trace argument. It closes #12.

My understanding is that, roughly, posterior SBC simply means treating the posterior distribution as the new prior, so most code should be the same (and it would make sense to put everything in the same SBC class). However, I don't know if I like the API.

Edit: I just realized I messed up the commit history between branches of this and the other active PR.

Checklist

• Code style is correct (follows ruff and black guidelines)
• Includes new or updated tests to cover the new feature
• New features are properly documented (with an example if appropriate)

[codecov-commenter]

Codecov Report

❌ Patch coverage is 98.27586% with 2 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@fe76da4). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
simuk/sbc.py	96.82%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #49   +/-   ##
=======================================
  Coverage        ?   96.19%           
=======================================
  Files           ?        3           
  Lines           ?      342           
  Branches        ?        0           
=======================================
  Hits            ?      329           
  Misses          ?       13           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[aloctavodia]

I have not checked the code yet. But notice that to treat the posterior as the prior, we can not simply plug the posterior "into the prior", because we don't usually have conjugate models. So in the paper they discuss the notion of augmented posterior, they concatenate observations to synthetic/replicated data. In the github repo there are a couple of examples in R (Stan) and python (bayesflow). https://github.com/TeemuSailynoja/posterior-sbc/tree/main/hierarchical-model

[currocam]

I see, that's definitely not what I did. I will mark it as draft and rethink it.

[aloctavodia]

I think here you have the missing step, i.e., concatenating the observations.

[currocam]

Hi @aloctavodia ,
Sorry for the late reply. I’ve been busy lately. I’ve been checking what I did, and the PR is far from being complete.

I’ve managed to produce these augmented_datasets, but I’m not sure how to actually fit these models conditioning on new data that has a different shape (twice the number of observations). The current version uses pm.observe that I believe does not allow changes in shape. The two main options that come up to mind are to:

• Expose a different interface that takes a factory function of models
• Require that data is encapsulated in some kind of mutable container (e.g. pm.Data?)

Is there a (better) third option?

[aloctavodia]

pm.Data should work for PyMC models. Not sure what we need to do for Bambi. Currently, we support Bambi by simply extracting the PyMC model. But it's ok to advance with PyMC and then see what to do with Bambi.

[aloctavodia]

Hey @currocam, just checking the status of this PR. Are you still working on this? Would you like some help?

[currocam]

Hi,
I’m trying to remember what exactly it was that I was stuck with. No, I’m not actively working on it now. I ended up coding the boilerplate for my own posterior SBC for my own projects and didn’t find time to wrap this PR up. Some help would be great yes. I'd prefer to rely on this package in the long run :=)

[aloctavodia]

I will try to help with this in the next two weeks.