Asking feedback for prey predator estimation (with lots of plots!)

[hyunjimoon]

May I ask for brief feedback on this notebook? For instance, what explanation and plot do you think I can add that can be helpful for those not used to inference diagnostics (seminar students). Full list of possible plots here:

@jandraor shared avoidable error paper with the following quote which would be a perfect ingredient to add to our #76.

• the validity of the statistical estimation procedure hinges on the independence of sequential measurement errors, which is clearly violated when observations are accumulated through time”
• models should be fit to raw, disaggregated data whenever possible and never to temporally accumulated data;
• when model assumptions, such as independence of errors, must be violated, careful checks for the effects of such violations should be performed;
• forecasts based on deterministic models, being by nature incapable of accurately communicating uncertainty, should be avoided
• stochastic models should be preferred to deterministic models in most circumstances because they afford improved accounting for real variability and increased opportunity for quantifying uncertainty. Post hoc comparison of simulated and actual data is a powerful and general procedure that can be used to distinguish model misspecification from real stochasticity.

[jandraor]

1. I simulate the model with some samples from the posterior and compare them against the data. This gives me a visual check of a "good fit".

2. Pair Plot I find it very useful to evaluate the shape of the joint posterior distribution. Add correlations for more information. Lots of correlations indicate a difficult parameter space.

3. Trace plots.

4. For marginal distributions, I found it useful to overlap prior & posterior histograms. This plot tells how much information we gained from the inference process for a particular parameter.

[hyunjimoon]

What makes you think it is a bug? Orange and blue are for the prey and red and green are for the predator.

[hyunjimoon]

Plot with a larger datapoint seems a little better.

[tomfid]

In the previous chart, it's oscillating with a period that appears to be 3*TIME STEP. I can't think of a legit way to generate something like that from the pred-prey model.

New chart seems much better.

[hyunjimoon]

TL;DR no problem in data, but problem in predictive plot code.

oscillating with a period that appears to be 3*TIME STEP
Good observation.. that 3 (surprisingly) seem to be R (number of region). I ran R = 1 and 3 and time axis wrongly increased by three. Stanify stacked all spatial heterogeneity into temporal one!

As you pointed out last week, bug seems to be coming from stacking (chain, draw) as (prior_draw) in draws2data().

[tomfid]

Nice detective work! Good that it's not really a bug so much as a benign but confusing display choice.