Add counterfactual terminology callouts to quasi-experimental notebooks#854
Add counterfactual terminology callouts to quasi-experimental notebooks#854drbenvincent wants to merge 11 commits intomainfrom
Conversation
Each of the four quasi-experimental notebooks (ITS, DiD, RDD, excess deaths) now has an admonition callout clarifying that "counterfactual" is used in the Rubin/potential outcomes sense, how this differs from Pearl's L3 unit-level counterfactuals, and why the usage is appropriate. Also removes a misleading do-operator reference from excess_deaths.ipynb and adds three bibliography entries (Rubin 1974, Imbens & Rubin 2015, Pearl 2009). Closes #852 Made-with: Cursor
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Made-with: Cursor
np.zeros/np.ones produce float64 by default, but the model's treated container expects bool. Pass dtype=bool to match the original data type. Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
Anticipates PR #850 renaming counterfactuals_do_operator to interventional_what_if_do_operator. Both PRs should merge together. Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
|
The individual call out boxes seem like a good idea. But i don't think Pearl's actual terminology here is "out of the box" helpful. A couple of things: "Abductive inference" is better than "abduction" as it's less easy to confuse with kidnapping. Secondly i don't think "abductive inference" is a great description of what Pearl is doing here. Or at least if we use the term we need to unpack it a bit more. Something like Pearlian counterfactuals aim at Inference about the best explanation of individual action. These depends on a thorough estimate of each individual's response profile. To do this we need to estimate their precise error term and re-use it in downstream calculations. " which require abduction — inferring unit-specific exogenous variables from observed data and then reasoning about what would have happened to that particular unit under a different action." I think i'd just drop the abduction term and emphasise it's about unit-specific effects. So in these cases distinct from what we care about DiD etc... |
NathanielF
left a comment
NathanielF
left a comment
There was a problem hiding this comment.
Maybe drop or explain abduction a bit better why Pearlian counterfactuals are effectively not relevant
…l-terminology-callouts
The previous re-execute (commits a71b4f7, 6bcfcba) produced nbformat-invalid notebooks: stream outputs were missing the required `name` field and display_data outputs were missing the required `metadata` field. This caused myst-nb to crash on Read the Docs with `AttributeError: name` while reading difference_in_differences. Re-executes all four affected notebooks against a current PyMC env using jupyter nbconvert, producing valid nbformat output. While re-executing, migrates two deprecated APIs to keep the notebooks runnable on PyMC 5.28+: - pm.MutableData -> pm.Data (DiD and excess_deaths) - model.add_coord(..., mutable=False) -> model.add_coord(...) (excess_deaths) The cross-references to interventional_what_if_do_operator and the L2/L3 counterfactual callouts added by this PR are preserved. Co-authored-by: Cursor <cursoragent@cursor.com>
…t-level Per review feedback, the previous wording leaned on Pearl's "abduction" / Level 3 terminology to set up a contrast it then had to talk itself out of. That framing was easy to misread and not a great description of what these methods actually deliver. Replaces the Pearl L2/L3 contrast paragraph with a single paragraph that frames the distinction more directly: these quasi-experimental methods target group-level average counterfactuals, which is distinct from unit-level counterfactuals about a particular individual. Pearl's hierarchy is mentioned only in passing, with the cross-reference to the interventional_what_if_do_operator notebook preserved for readers who want the formal treatment. Applies the same revision to all four notebooks (DiD, ITS, RDD, excess_deaths) and their .myst.md pairs. Co-authored-by: Cursor <cursoragent@cursor.com>
|
View / edit / reply to this conversation on ReviewNB aloctavodia commented on 2026-05-04T05:43:01Z Maybe something like this We rely on the parallel trends assumption, which states that in the absence of treatment, the treated and control groups would have followed the same trajectory over time. This allows us to use the control group's...
|
|
View / edit / reply to this conversation on ReviewNB aloctavodia commented on 2026-05-04T05:43:03Z ... We extrapolate pre-intervention trends to predict the unobserved potential outcome Y(0): what would have happened, on average, without the intervention. This is a group-level counterfactual prediction, standard in the quasi-experimental literature. This group-level counterfactual is distinct from a unit-level... |
2 participants
Summary
:::{admonition}callout to four quasi-experimental notebooks (interrupted_time_series,difference_in_differences,regression_discontinuity,excess_deaths) clarifying that "counterfactual" is used in the Rubin/potential outcomes sense and how this differs from Pearl's L3 unit-level counterfactualsexcess_deaths.ipynb(the notebook does not usepm.do)dtypebug inregression_discontinuity.ipynb(np.zeros/np.onesproducing float64 instead of bool for thetreateddata container)pm.MutableData→pm.Data(difference_in_differences,excess_deaths)model.add_coord(..., mutable=False)→model.add_coord(...)(excess_deaths)jupyter nbconvert --executeso outputs are nbformat-valid (the previous re-execute had stripped requiredname/metadatafields, which crashedmyst-nbon Read the Docs)Closes #852
Changes Made
Each callout is tailored to the specific method:
All callouts include references and cross-reference the
interventional_what_if_do_operatornotebook for the full L2/L3 distinction.Test plan
.ipynband.myst.mdversions updated consistently