Skip to content

Commit bfce5c6

Browse files
drbenvincentdrbenvincent
committed
minor restructure of notebook for clarity, readability, and flow
1 parent f37eef9 commit bfce5c6

File tree

1 file changed

+88
-69
lines changed

1 file changed

+88
-69
lines changed

‎docs/source/notebooks/graded_intervention_time_series_single_channel_ols.ipynb‎

Lines changed: 88 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,8 @@
3535
"\n",
3636
"### The Autocorrelation Challenge\n",
3737
"\n",
38+
"#### Introduction: Understanding the Problem\n",
39+
"\n",
3840
"**Autocorrelation** occurs when observations in a time series are correlated with their own past values. In causal inference with time series data, this creates a fundamental challenge:\n",
3941
"\n",
4042
"**What is autocorrelation?**\n",
@@ -66,50 +68,69 @@
6668
"4. **Bayesian time series models**: Full posterior inference with temporal dependencies\n",
6769
"5. **Bootstrap methods**: Resample with preserved temporal structure\n",
6870
"\n",
69-
"**This implementation uses HAC standard errors** because they offer an excellent balance of:\n",
71+
"This implementation provides both **HAC** and **ARIMAX** approaches, each with distinct advantages for different use cases.\n",
72+
"\n",
73+
"#### Approach 1: HAC Standard Errors (Default)\n",
74+
"\n",
75+
"**HAC (Heteroskedasticity and Autocorrelation Consistent) standard errors** — also known as **Newey-West standard errors** {cite:p}`newey1987simple` — provide robust inference by correcting standard errors without requiring specification of the autocorrelation structure.\n",
76+
"\n",
77+
"**Advantages:**\n",
7078
"- **Simplicity**: No need to specify autocorrelation structure (order of AR/MA terms)\n",
7179
"- **Robustness**: Works with any autocorrelation pattern (not just AR or MA)\n",
7280
"- **Computational efficiency**: Fast OLS with corrected standard errors\n",
73-
"- **Proven reliability**: Well-established method with strong theoretical properties {cite:p}`newey1987simple`\n",
81+
"- **Proven reliability**: Well-established method with strong theoretical properties\n",
7482
"\n",
7583
"With HAC (see detailed explanation in the admonition box below):\n",
7684
"- ✅ **Causal estimates remain valid**: Treatment effect coefficients are unbiased\n",
7785
"- ✅ **Inference is corrected**: Standard errors, confidence intervals, and p-values account for autocorrelation\n",
7886
"- ✅ **No model specification required**: Don't need to guess AR order or lag structure\n",
7987
"- ✅ **Honest uncertainty quantification**: Confidence intervals reflect true uncertainty\n",
8088
"\n",
81-
"The tradeoff: HAC standard errors are **wider** (more conservative) than naive OLS, but they provide **trustworthy inference** even when residuals show complex autocorrelation patterns.\n",
89+
"**Tradeoff:** HAC standard errors are **wider** (more conservative) than naive OLS, but they provide **trustworthy inference** even when residuals show complex autocorrelation patterns.\n",
90+
"\n",
91+
"**This notebook demonstrates HAC inference** in the main analysis sections, showing how it compares to naive OLS and why it matters for valid causal inference.\n",
92+
"\n",
93+
"::::{admonition} Understanding HAC Standard Errors\n",
94+
":class: note\n",
95+
"\n",
96+
"Time series data typically violates OLS assumptions because:\n",
97+
"- **Autocorrelation**: Past values influence current values (e.g., yesterday's weather affects today's, habits persist over weeks)\n",
98+
"- **Heteroskedasticity**: Variance changes over time (e.g., more volatility in certain seasons)\n",
99+
"\n",
100+
"When these violations occur, OLS **coefficient estimates remain unbiased**, but **standard errors are incorrect** — typically too small, leading to overconfident inference (narrow confidence intervals, artificially low p-values).\n",
101+
"\n",
102+
"**HAC (Heteroskedasticity and Autocorrelation Consistent) standard errors** — also known as **Newey-West standard errors** {cite:p}`newey1987simple` — provide robust inference by correcting standard errors for these violations. This gives reliable confidence intervals and hypothesis tests even when residuals are correlated.\n",
103+
"\n",
104+
"**Key Parameter:**\n",
105+
"- `hac_maxlags`: Controls how many periods of autocorrelation to account for. CausalPy uses the Newey-West rule of thumb: `floor(4*(n/100)^(2/9))`. For our 104-week dataset, this gives `hac_maxlags=4`, accounting for up to 4 weeks of residual correlation.\n",
106+
"\n",
107+
"**Tradeoff:** HAC standard errors are wider (more conservative) than naive OLS, but provide honest uncertainty quantification for time series data.\n",
108+
"::::\n",
109+
"\n",
110+
"#### Approach 2: ARIMAX Models\n",
111+
"\n",
112+
"**ARIMAX (ARIMA with eXogenous variables)** explicitly models the autocorrelation structure of residuals using ARIMA(p,d,q) processes, following the classical Box & Tiao (1975) intervention analysis framework {cite:p}`box1975intervention`.\n",
113+
"\n",
114+
"**Advantages:**\n",
115+
"- **Efficiency**: Smaller standard errors when ARIMA structure is correctly specified\n",
116+
"- **Classical methodology**: Follows the original intervention analysis approach\n",
117+
"- **Explicit error modeling**: Can characterize and forecast residual dynamics\n",
82118
"\n",
83-
"**Future extensions** could add ARIMAX or Bayesian models for explicit temporal structure modeling, but HAC provides an excellent starting point for practical causal inference with time series data.\n",
119+
"**Tradeoffs:**\n",
120+
"- **Requires specification**: Must choose p, d, q orders (typically via ACF/PACF plots)\n",
121+
"- **Misspecification risk**: Wrong orders can lead to biased or inefficient inference\n",
122+
"- **Less robust**: More sensitive to outliers and structural breaks\n",
84123
"\n",
85-
"This notebook demonstrates why autocorrelation-robust inference matters by:\n",
86-
"1. Simulating data with realistic autocorrelation (AR(1) errors)\n",
87-
"2. Fitting the TF-ITS model with HAC standard errors\n",
88-
"3. Comparing naive vs HAC inference to show the concrete difference\n",
89-
"\n"
124+
"**Section 5 of this notebook demonstrates ARIMAX** as an alternative error model, comparing it to HAC and providing guidance on when to use each approach."
90125
]
91126
},
92127
{
93128
"cell_type": "markdown",
94129
"metadata": {},
95130
"source": [
96-
"## Example Scenario: Water Restrictions Policy\n",
97-
"\n",
98-
"A regional water authority in a dry climate implements a **drought-responsive communications policy**. Communication intensity (0-10 scale) varies based on cumulative rainfall deficit over the past 6 weeks. During sustained drought, messaging ramps up to encourage conservation. Most of the time, communications are zero (no routine messaging).\n",
99-
"\n",
100-
"Why this example demonstrates TF-ITS strengths:\n",
101-
"- **Graded intervention**: Communication intensity varies from 0-10, not on/off\n",
102-
"- **Sparse activation**: Policy activates only during drought (realistic, cost-effective)\n",
103-
"- **Saturation**: Repeated messages have diminishing returns as people become desensitized\n",
104-
"- **Adstock**: Behavioral habits change gradually and persist over weeks\n",
105-
"- **Confounders**: Temperature and rainfall directly affect water consumption and must be controlled\n",
131+
"## Notebook Overview\n",
106132
"\n",
107-
"While we use water policy, this method applies to any domain with graded interventions and carryover effects:\n",
108-
"- Public health campaigns (vaccination messaging, smoking cessation)\n",
109-
"- Marketing mix modeling (advertising spend, promotions) \n",
110-
"- Environmental policy (emissions reduction programs)\n",
111-
"- Traffic management (congestion pricing communications)\n",
112-
"- Education interventions (remediation program intensity)"
133+
"This notebook demonstrates Graded Intervention Time Series (Transfer Function ITS) analysis using a simulated water consumption dataset. We'll walk through data simulation, model fitting with transform parameter estimation, diagnostic checks, counterfactual analysis, and a comparison of different approaches to handling autocorrelation in time series data (HAC vs ARIMAX error models)."
113134
]
114135
},
115136
{
@@ -119,35 +140,12 @@
119140
":::{admonition} Implementation notes\n",
120141
":class: warning\n",
121142
"This notebook demonstrates the **non-Bayesian implementation** using:\n",
122-
"- OLS regression with HAC standard errors (fast, robust inference)\n",
143+
"- OLS regression first with with HAC standard errors (fast, robust inference), then with ARIMAX.\n",
123144
"- Automated transform parameter estimation via grid search or continuous optimization\n",
124145
"- Point estimates only (future: bootstrap confidence intervals, Bayesian uncertainty quantification)\n",
125146
":::"
126147
]
127148
},
128-
{
129-
"cell_type": "markdown",
130-
"metadata": {},
131-
"source": [
132-
"::::{admonition} Understanding HAC Standard Errors\n",
133-
":class: note\n",
134-
"\n",
135-
"Time series data typically violates OLS assumptions because:\n",
136-
"- **Autocorrelation**: Past values influence current values (e.g., yesterday's weather affects today's, habits persist over weeks)\n",
137-
"- **Heteroskedasticity**: Variance changes over time (e.g., more volatility in certain seasons)\n",
138-
"\n",
139-
"When these violations occur, OLS **coefficient estimates remain unbiased**, but **standard errors are incorrect** — typically too small, leading to overconfident inference (narrow confidence intervals, artificially low p-values).\n",
140-
"\n",
141-
"**HAC (Heteroskedasticity and Autocorrelation Consistent) standard errors** — also known as **Newey-West standard errors** {cite:p}`newey1987simple` — provide robust inference by correcting standard errors for these violations. This gives reliable confidence intervals and hypothesis tests even when residuals are correlated.\n",
142-
"\n",
143-
"**Key Parameter:**\n",
144-
"- `hac_maxlags`: Controls how many periods of autocorrelation to account for. CausalPy uses the Newey-West rule of thumb: `floor(4*(n/100)^(2/9))`. For our 104-week dataset, this gives `hac_maxlags=4`, accounting for up to 4 weeks of residual correlation.\n",
145-
"\n",
146-
"**Tradeoff:** HAC standard errors are wider (more conservative) than naive OLS, but provide honest uncertainty quantification for time series data.\n",
147-
"\n",
148-
"::::\n"
149-
]
150-
},
151149
{
152150
"cell_type": "code",
153151
"execution_count": 1,
@@ -176,6 +174,29 @@
176174
"%config InlineBackend.figure_format = 'retina'"
177175
]
178176
},
177+
{
178+
"cell_type": "markdown",
179+
"metadata": {},
180+
"source": [
181+
"## Example Scenario: Water Restrictions Policy\n",
182+
"\n",
183+
"A regional water authority in a dry climate implements a **drought-responsive communications policy**. Communication intensity (0-10 scale) varies based on cumulative rainfall deficit over the past 6 weeks. During sustained drought, messaging ramps up to encourage conservation. Most of the time, communications are zero (no routine messaging).\n",
184+
"\n",
185+
"Why this example demonstrates TF-ITS strengths:\n",
186+
"- **Graded intervention**: Communication intensity varies from 0-10, not on/off\n",
187+
"- **Sparse activation**: Policy activates only during drought (realistic, cost-effective)\n",
188+
"- **Saturation**: Repeated messages have diminishing returns as people become desensitized\n",
189+
"- **Adstock**: Behavioral habits change gradually and persist over weeks\n",
190+
"- **Confounders**: Temperature and rainfall directly affect water consumption and must be controlled\n",
191+
"\n",
192+
"While we use water policy, this method applies to any domain with graded interventions and carryover effects:\n",
193+
"- Public health campaigns (vaccination messaging, smoking cessation)\n",
194+
"- Marketing mix modeling (advertising spend, promotions) \n",
195+
"- Environmental policy (emissions reduction programs)\n",
196+
"- Traffic management (congestion pricing communications)\n",
197+
"- Education interventions (remediation program intensity)\n"
198+
]
199+
},
179200
{
180201
"cell_type": "markdown",
181202
"metadata": {},
@@ -374,8 +395,6 @@
374395
"cell_type": "markdown",
375396
"metadata": {},
376397
"source": [
377-
"## 2. Visualize the Data\n",
378-
"\n",
379398
"Let's look at the water consumption and communication intensity time series. Notice:\n",
380399
"- **Very dry climate** with extended zero-rainfall periods in summer\n",
381400
"- **Communications are zero most of the time** - only activated during sustained drought\n",
@@ -471,7 +490,7 @@
471490
"cell_type": "markdown",
472491
"metadata": {},
473492
"source": [
474-
"## 3. Model Fitting\n",
493+
"## 2. Model Fitting\n",
475494
"\n",
476495
"Fitting a transfer function model involves finding both the optimal transform parameters and the regression coefficients. This is accomplished through a nested optimization procedure. In the outer loop, the algorithm searches for the best saturation and adstock parameters—either by exhaustively evaluating all combinations on a discrete grid, or by using continuous optimization to search more efficiently through the parameter space. For each candidate set of transform parameters, the inner loop applies these transformations to the raw treatment variable and fits a regression model (OLS or ARIMAX) to the data. The root mean squared error (RMSE) of each fitted model is computed, and the parameter combination that minimizes this error is selected.\n",
477496
"\n",
@@ -533,7 +552,7 @@
533552
"cell_type": "markdown",
534553
"metadata": {},
535554
"source": [
536-
"## 4. Visualize Estimated vs True Transform Parameters\n",
555+
"## 3. Visualize Estimated vs True Transform Parameters\n",
537556
"\n",
538557
"Since we know the true parameters used to generate the data, we can compare the estimated transforms to the true transforms. This helps us assess **parameter recovery** - how well the estimation procedure identifies the true data-generating process.\n",
539558
"\n",
@@ -626,11 +645,11 @@
626645
"cell_type": "markdown",
627646
"metadata": {},
628647
"source": [
629-
"## 5. Model Methods and Diagnostics\n",
648+
"## 4. Model Methods and Diagnostics\n",
630649
"\n",
631650
"Now that we have a fitted model with estimated transforms, let's explore the available methods for analysis and diagnostics.\n",
632651
"\n",
633-
"### 5.1 Model Summary\n",
652+
"### Model Summary\n",
634653
"\n",
635654
"View the fitted model coefficients and their **HAC standard errors** (robust to autocorrelation and heteroskedasticity):\n"
636655
]
@@ -674,7 +693,7 @@
674693
"cell_type": "markdown",
675694
"metadata": {},
676695
"source": [
677-
"### 5.2 Model Fit Visualization\n",
696+
"### Model Fit Visualization\n",
678697
"\n",
679698
"Plot observed vs fitted values and residuals:\n"
680699
]
@@ -710,7 +729,7 @@
710729
"cell_type": "markdown",
711730
"metadata": {},
712731
"source": [
713-
"### 5.3 Residual Diagnostics\n",
732+
"### Residual Diagnostics\n",
714733
"\n",
715734
"Check for autocorrelation in residuals using ACF/PACF plots and Ljung-Box test. \n",
716735
"\n",
@@ -769,7 +788,7 @@
769788
"cell_type": "markdown",
770789
"metadata": {},
771790
"source": [
772-
"### 5.4 Why HAC Standard Errors Matter\n",
791+
"### Why HAC Standard Errors Matter\n",
773792
"\n",
774793
"The diagnostics above show **significant residual autocorrelation** (Ljung-Box test p-values < 0.05). This is **expected and realistic** even with a well-specified model! \n",
775794
"\n",
@@ -998,7 +1017,7 @@
9981017
"cell_type": "markdown",
9991018
"metadata": {},
10001019
"source": [
1001-
"### 5.5 Impulse Response Function\n",
1020+
"### Impulse Response Function\n",
10021021
"\n",
10031022
"Visualize how communication effects persist over time through the adstock transformation:\n"
10041023
]
@@ -1034,7 +1053,7 @@
10341053
"cell_type": "markdown",
10351054
"metadata": {},
10361055
"source": [
1037-
"### 5.6 Counterfactual Effect Estimation\n",
1056+
"### Counterfactual Effect Estimation\n",
10381057
"\n",
10391058
"Estimate the effect of the communications policy by comparing observed outcomes to a counterfactual where communications were never implemented:\n"
10401059
]
@@ -1076,7 +1095,7 @@
10761095
"cell_type": "markdown",
10771096
"metadata": {},
10781097
"source": [
1079-
"## 6. Alternative Error Model: ARIMAX\n",
1098+
"## 5. Alternative Error Model: ARIMAX\n",
10801099
"\n",
10811100
"So far we've used **HAC (Newey-West) standard errors**, which provide robust inference without requiring us to specify the autocorrelation structure. This is the recommended default approach.\n",
10821101
"\n",
@@ -1104,7 +1123,7 @@
11041123
"cell_type": "markdown",
11051124
"metadata": {},
11061125
"source": [
1107-
"### 6.1 Fit Model with ARIMAX Errors\n",
1126+
"### Fit Model with ARIMAX Errors\n",
11081127
"\n",
11091128
"Since we generated the data with AR(1) errors (`rho_error = 0.5`), we know the true error structure is ARIMA(1,0,0). Let's fit the model with ARIMAX using this specification:\n"
11101129
]
@@ -1194,7 +1213,7 @@
11941213
"cell_type": "markdown",
11951214
"metadata": {},
11961215
"source": [
1197-
"### 6.2 Model Fit Visualization\n",
1216+
"### Model Fit Visualization\n",
11981217
"\n",
11991218
"Let's visualize the ARIMAX model fit to see how well it captures the data patterns:\n"
12001219
]
@@ -1236,7 +1255,7 @@
12361255
"cell_type": "markdown",
12371256
"metadata": {},
12381257
"source": [
1239-
"### 6.3 Residual Diagnostics\n",
1258+
"### Residual Diagnostics\n",
12401259
"\n",
12411260
"A key advantage of ARIMAX is that by explicitly modeling the autocorrelation structure, the residuals should exhibit less autocorrelation. Let's check this with ACF/PACF plots and the Ljung-Box test:\n"
12421261
]
@@ -1297,7 +1316,7 @@
12971316
"cell_type": "markdown",
12981317
"metadata": {},
12991318
"source": [
1300-
"### 6.4 Impulse Response Function\n",
1319+
"### Impulse Response Function\n",
13011320
"\n",
13021321
"The impulse response function visualizes how a one-unit increase in communication intensity affects water consumption dynamically over time, accounting for the adstock effect:\n"
13031322
]
@@ -1339,7 +1358,7 @@
13391358
"cell_type": "markdown",
13401359
"metadata": {},
13411360
"source": [
1342-
"### 6.5 Counterfactual Effect Estimation\n",
1361+
"### Counterfactual Effect Estimation\n",
13431362
"\n",
13441363
"We can estimate the total causal effect of the communications policy by comparing observed outcomes to a counterfactual scenario where communications were never implemented:\n"
13451364
]
@@ -1455,9 +1474,9 @@
14551474
"cell_type": "markdown",
14561475
"metadata": {},
14571476
"source": [
1458-
"### 6.2 Comparison: HAC vs ARIMAX\n",
1477+
"## Comparison: HAC vs ARIMAX\n",
14591478
"\n",
1460-
"Let's compare the two approaches side-by-side to understand their differences:\n"
1479+
"Let's compare the two approaches side-by-side to understand their differences:"
14611480
]
14621481
},
14631482
{
@@ -1644,7 +1663,7 @@
16441663
"cell_type": "markdown",
16451664
"metadata": {},
16461665
"source": [
1647-
"### 6.3 Decision Guide: Which Error Model to Use?\n",
1666+
"### Decision Guide: Which Error Model to Use?\n",
16481667
"\n",
16491668
"Here's a practical guide for choosing between HAC and ARIMAX:\n",
16501669
"\n",

0 commit comments

Comments
 (0)