|
33 | 33 | "\n", |
34 | 34 | "In this notebook we'll show why we should be careful attempting to model the joint-distribution of the propensity score and the outcome variable, but still make good use of the propensity score. \n", |
35 | 35 | "\n", |
36 | | - "### Brief Digression on the Mathematics\n", |
| 36 | + "#### Brief Digression on the Mathematics\n", |
37 | 37 | "\n", |
38 | 38 | "Consider that we have the following three variables:\n", |
39 | 39 | "\n", |
|
3785 | 3785 | "cell_type": "markdown", |
3786 | 3786 | "metadata": {}, |
3787 | 3787 | "source": [ |
3788 | | - "## Nets Example\n", |
| 3788 | + "### Nets Example\n", |
3789 | 3789 | "\n", |
3790 | 3790 | "Next we'll asses a data set used by Andrew Heiss to demonstrate propensity score methods with `brms`. " |
3791 | 3791 | ] |
|
5439 | 5439 | "cell_type": "markdown", |
5440 | 5440 | "metadata": {}, |
5441 | 5441 | "source": [ |
5442 | | - "## LaLonde Example\n", |
| 5442 | + "### LaLonde Example\n", |
5443 | 5443 | "\n", |
5444 | 5444 | "The Lalonde Data set is famous because it highlights a problem with naive causal contrasts. It is discussed by Angrist and Pischke in their _Mostly Harmless Econometrics_ as an example of how regression controls can tolerably address selection effects in a way similar to propensity score weighting. So we should hope the a well specified outcome model can identify the treatment effects plausibly here too. " |
5445 | 5445 | ] |
|
6133 | 6133 | "cell_type": "markdown", |
6134 | 6134 | "metadata": {}, |
6135 | 6135 | "source": [ |
6136 | | - "## NHEFS \n", |
| 6136 | + "### NHEFS \n", |
6137 | 6137 | "\n", |
6138 | 6138 | "Finally we turn to the NHEFS data. This data is known to be have a complex covariate profile for measuring aspects smokers health. We might suspect that there is some unmeasured confounding in this data set that would be hard to pick up on with simple regression controls. " |
6139 | 6139 | ] |
|
6733 | 6733 | "cell_type": "markdown", |
6734 | 6734 | "metadata": {}, |
6735 | 6735 | "source": [ |
6736 | | - "## Two-Stage Outcome Modelling with CausalPy\n" |
| 6736 | + "### Two-Stage Outcome Modelling with CausalPy\n", |
| 6737 | + "\n", |
| 6738 | + "Next we show how to achieve these steps with the simpler CausalPy experiment API. " |
6737 | 6739 | ] |
6738 | 6740 | }, |
6739 | 6741 | { |
|
6796 | 6798 | "cell_type": "markdown", |
6797 | 6799 | "metadata": {}, |
6798 | 6800 | "source": [ |
6799 | | - "### Comparing Inverse Propensity Score Weighting and Covariate Adjustment" |
| 6801 | + "### Comparing Inverse Propensity Score Weighting and Covariate Adjustment\n", |
| 6802 | + "\n", |
| 6803 | + "The two step procedure doesn't jusst apply for regression adjustment methods as we've seen here, but can be used to apply inverse weighting techniques too. " |
6800 | 6804 | ] |
6801 | 6805 | }, |
6802 | 6806 | { |
|
6829 | 6833 | "result.plot_ate(result.idata);" |
6830 | 6834 | ] |
6831 | 6835 | }, |
| 6836 | + { |
| 6837 | + "cell_type": "markdown", |
| 6838 | + "metadata": {}, |
| 6839 | + "source": [ |
| 6840 | + "which can be compared against the two-step regression adjustment here. " |
| 6841 | + ] |
| 6842 | + }, |
6832 | 6843 | { |
6833 | 6844 | "cell_type": "code", |
6834 | 6845 | "execution_count": 34, |
|
6859 | 6870 | ")" |
6860 | 6871 | ] |
6861 | 6872 | }, |
| 6873 | + { |
| 6874 | + "cell_type": "markdown", |
| 6875 | + "metadata": {}, |
| 6876 | + "source": [ |
| 6877 | + "Yielding similar, but not identical results. " |
| 6878 | + ] |
| 6879 | + }, |
6862 | 6880 | { |
6863 | 6881 | "cell_type": "code", |
6864 | 6882 | "execution_count": 35, |
|
7144 | 7162 | "cell_type": "markdown", |
7145 | 7163 | "metadata": {}, |
7146 | 7164 | "source": [ |
7147 | | - "## Conclusion: It' the Model Stupid!\n", |
| 7165 | + "### Conclusion: Modularity as Causal Discipline\n", |
| 7166 | + "\n", |
| 7167 | + "When attempting to estimate treatment effects using Bayesian inference, a natural but risky strategy is to fit a joint model for both the treatment assignment and the outcome. That is, to specify a full model and infer the parameters of both components simultaneously.\n", |
7148 | 7168 | "\n", |
| 7169 | + "However, this joint approach introduces a feedback loop: the outcome $Y$ can influence the estimation of the treatment mechanism $P(T | X)$. This violates the original logic of design-based inference, where treatment assignment should be modeled independently of the observed outcomes. This phenomenon is often subtle but can lead to biased treatment effect estimates.\n", |
7149 | 7170 | "\n", |
| 7171 | + "Across several examples, we have shown that fitting a full joint model distorts the treatment effect estimate relative to a two-step (modular) approach.\n", |
| 7172 | + "In other cases, joint and modular approaches yield nearly identical estimates — usually when the treatment mechanism is well-identified from covariates alone. With these observations in scope, we recommend that practioners generally follow a two-step or modular approach. Either two-stage inverse propensity score weighting or regression adjustment with the propensity score as an additional covariate. Both methods are available now in `CausalPy`. \n", |
7150 | 7173 | "\n", |
| 7174 | + "Framed this way we can see that joint model violates the temporal precedence of the treatment assignment and outcome process. The 2-stage Bayesian procedures ensure that the causal ordering encoded in the actual data generating process is respected in the estimation process. The confounding adjustment achieved with propensity score must occur without access to information about the outcome. A well-specified propensity score model can substantially improve causal estimates (as we've seen), especially when the outcome model is weak or mis-specified. Propensity scores do not only serve to reduce dimensionality; they formalize the treatment mechanism and encode information that the outcome model might fail to recover. This explains their continued prominence in modern causal inference.\n", |
7151 | 7175 | "\n", |
7152 | | - "## References\n", |
| 7176 | + "### References\n", |
7153 | 7177 | ":::{bibliography}\n", |
7154 | 7178 | ":filter: docname in docnames\n", |
7155 | 7179 | ":::" |
7156 | 7180 | ] |
| 7181 | + }, |
| 7182 | + { |
| 7183 | + "cell_type": "markdown", |
| 7184 | + "metadata": {}, |
| 7185 | + "source": [] |
7157 | 7186 | } |
7158 | 7187 | ], |
7159 | 7188 | "metadata": { |
|
0 commit comments