You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/case_studies/bayesian_sem_workflow.ipynb
+27-3Lines changed: 27 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -5754,9 +5754,9 @@
5754
5754
"id": "e51eaf54",
5755
5755
"metadata": {},
5756
5756
"source": [
5757
-
"#### Parameter Recovery Plots\n",
5757
+
"#### The Parameter Recovery Process\n",
5758
5758
"\n",
5759
-
"But we can actually assess the degree of parameter recovery because we know the true values"
5759
+
"But we can actually assess the degree of parameter recovery because we know the true values. This is a pivotal part of the model building process, akin to how writer's read aloud their own work to test it for assonance, cogency and flow. In simulation based probabilistic modelling we should be able generate data from models with known parameters, and recover the latent parameter values through the inferential workflow. "
5760
5760
]
5761
5761
},
5762
5762
{
@@ -5817,12 +5817,26 @@
5817
5817
");"
5818
5818
]
5819
5819
},
5820
+
{
5821
+
"cell_type": "markdown",
5822
+
"id": "41e491be",
5823
+
"metadata": {},
5824
+
"source": [
5825
+
"Here we see how the posterior distributions “recover” the true values within uncertainty ensuring the model is faithful to the data generating process. Were the effort at parameter recover to fail, we would equally have learned something about our model. Parameter recovery exercises helps discover issues of mis-specification or unidentified parameters. Put another way, they tell us how informative our data is with respect to our data generating model. Verlyn Klinkenborg starts his justly famous book _Several short sentences about writing_ with the following advice: \n",
5826
+
"\n",
5827
+
"> \"Here, in short, is what i want to tell you. Know what each sentence says, What it doesn't say, And what it implies. Of these, the hardest is know what each sentence actually says\" - V. Klinkenborg\n",
5828
+
"\n",
5829
+
"This advice transfers exactly to the art of statistical modelling. To know what our model says, we need to say it aloud. We need to feel how it lands with an audience. We need to understand is implications and limitations. The Bayesian workflow explores the depths of meaning achieved by our statistical approximations. It traces out the effects of interlocking components and the layered interactions of structural regressions. In each articulation we're testing which the flavours of reality resonate in the telling. What shape the posterior? How plausible the range of values? How faithful are our predictions to reality? On these questions we weigh each model just as the writer weighs each sentence for their effects. "
5830
+
]
5831
+
},
5820
5832
{
5821
5833
"cell_type": "markdown",
5822
5834
"id": "bf4b62e5",
5823
5835
"metadata": {},
5824
5836
"source": [
5825
-
"### Hypothesis Evaluation: How does group membership change direct effects?"
5837
+
"### Hypothesis Evaluation: How does group membership change direct effects?\n",
5838
+
"\n",
5839
+
"In this case we've encoded a differnce in the regression effects for each of the two groups. These effects are easily recovered with quantifiable measures of uncertainty around the treatment effects."
5826
5840
]
5827
5841
},
5828
5842
{
@@ -5859,6 +5873,16 @@
5859
5873
");"
5860
5874
]
5861
5875
},
5876
+
{
5877
+
"cell_type": "markdown",
5878
+
"id": "59c4d17a",
5879
+
"metadata": {},
5880
+
"source": [
5881
+
"In an applied setting it's these kinds of implications that are crucially important to surface and understand. From a workflow point of view we want to ensure that our modelling drives clarity on these precise points and avoids adding noise generally. \n",
5882
+
"\n",
5883
+
"Another way, we might interrogate the implications of a model is to see how well it can predict \"downstream\" outcomes of the implied model. In the job-satisfaction setting we might wonder about how job-satisfaction relates to attrition risk?"
But we can actually assess the degree of parameter recovery because we know the true values
1190
+
But we can actually assess the degree of parameter recovery because we know the true values. This is a pivotal part of the model building process, akin to how writer's read aloud their own work to test it for assonance, cogency and flow. In simulation based probabilistic modelling we should be able generate data from models with known parameters, and recover the latent parameter values through the inferential workflow.
1191
1191
1192
1192
```{code-cell} ipython3
1193
1193
az.plot_posterior(
@@ -1203,8 +1203,18 @@ az.plot_posterior(
1203
1203
);
1204
1204
```
1205
1205
1206
+
Here we see how the posterior distributions “recover” the true values within uncertainty ensuring the model is faithful to the data generating process. Were the effort at parameter recover to fail, we would equally have learned something about our model. Parameter recovery exercises helps discover issues of mis-specification or unidentified parameters. Put another way, they tell us how informative our data is with respect to our data generating model. Verlyn Klinkenborg starts his justly famous book _Several short sentences about writing_ with the following advice:
1207
+
1208
+
> "Here, in short, is what i want to tell you. Know what each sentence says, What it doesn't say, And what it implies. Of these, the hardest is know what each sentence actually says" - V. Klinkenborg
1209
+
1210
+
This advice transfers exactly to the art of statistical modelling. To know what our model says, we need to say it aloud. We need to feel how it lands with an audience. We need to understand is implications and limitations. The Bayesian workflow explores the depths of meaning achieved by our statistical approximations. It traces out the effects of interlocking components and the layered interactions of structural regressions. In each articulation we're testing which the flavours of reality resonate in the telling. What shape the posterior? How plausible the range of values? How faithful are our predictions to reality? On these questions we weigh each model just as the writer weighs each sentence for their effects.
1211
+
1212
+
+++
1213
+
1206
1214
### Hypothesis Evaluation: How does group membership change direct effects?
1207
1215
1216
+
In this case we've encoded a differnce in the regression effects for each of the two groups. These effects are easily recovered with quantifiable measures of uncertainty around the treatment effects.
In an applied setting it's these kinds of implications that are crucially important to surface and understand. From a workflow point of view we want to ensure that our modelling drives clarity on these precise points and avoids adding noise generally.
1231
+
1232
+
Another way, we might interrogate the implications of a model is to see how well it can predict "downstream" outcomes of the implied model. In the job-satisfaction setting we might wonder about how job-satisfaction relates to attrition risk?
0 commit comments