Skip to content

Commit 81a7155

Browse files
committed
Add diagnostic plots
1 parent 7f8b80b commit 81a7155

File tree

3 files changed

+5
-2
lines changed

3 files changed

+5
-2
lines changed

README.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -228,7 +228,6 @@ When feeding the data to our trained network, we almost immediately obtain poste
228228

229229
```python
230230
sim_preds = amortizer(sim_data)
231-
sim_preds[0,:]
232231
```
233232

234233
How good are these predicted probabilities? We can have a look at the calibration:
@@ -237,12 +236,16 @@ How good are these predicted probabilities? We can have a look at the calibratio
237236
cal_curves = bf.diagnostics.plot_calibration_curves(sim_indices, sim_preds)
238237
```
239238

240-
Our approximator shows excellent calibration, with an expected calibration error (ECE) close to 0 and most predicted probabilities being certain of the model underlying a data set. We can further assess patterns of misclassification with a confusion matrix:
239+
<img src="img/showcase_calibration_curves.png" width=65% height=65%>
240+
241+
Our approximator shows excellent calibration, with the calibration curve being closely aligned to the diagonal, an expected calibration error (ECE) near 0 and most predicted probabilities being certain of the model underlying a data set. We can further assess patterns of misclassification with a confusion matrix:
241242

242243
```python
243244
conf_matrix = bf.diagnostics.plot_confusion_matrix(sim_indices, sim_preds)
244245
```
245246

247+
<img src="img/showcase_confusion_matrix.png" width=65% height=65%>
248+
246249
For the vast majority of simulated data sets, the generating model is correctly detected. With these diagnostic results backing us up, we can safely apply our trained network to empirical data.
247250

248251
BayesFlow is also able to conduct model comparison for hierarchical models. See this [tutorial notebook](docs/source/tutorial_notebooks/Hierarchical_Model_Comparison_MPT.ipynb) for an introduction to the associated workflow.
181 KB
Loading

img/showcase_confusion_matrix.png

77.3 KB
Loading

0 commit comments

Comments
 (0)