misc formatting + text changes

Author: drbenvincent

docs/source/notebooks/its_lift_test.ipynb

@@ -78,21 +78,21 @@
     "\n",
     "This notebook is organized into the following sections:\n",
     "\n",
-    "1. **Setup and Imports**: Load required packages\n",
-    "2. **Part I: Data Simulation**: Generate synthetic marketing data with realistic patterns\n",
+    "* **Setup and Imports**: Load required packages\n",
+    "* **Part I: Data Simulation**: Generate synthetic marketing data with realistic patterns\n",
     "   - Define time periods and intervention dates\n",
     "   - Create media spend data for multiple channels\n",
     "   - Apply marketing transformations (adstock and saturation)\n",
     "   - Construct the target variable (sales)\n",
     "   \n",
     "   *Note: If you already have your own data and just want to learn how to evaluate lift, you can skip directly to Part II.*\n",
     "\n",
-    "3. **Part II: Interrupted Time Series Analysis**: Fit the ITS model and generate counterfactual predictions\n",
-    "4. **Part III: Calculating Lift Metrics**: \n",
+    "* **Part II: Interrupted Time Series Analysis**: Fit the ITS model and generate counterfactual predictions\n",
+    "* **Part III: Calculating Lift Metrics**: \n",
     "   - Absolute Lift (total incremental sales)\n",
     "   - Percentage Lift (% increase vs baseline)\n",
     "   - Return on Investment (ROI)\n",
-    "5. **Summary and Interpretation**: Business implications, assumptions, and next steps\n",
+    "* **Summary and Interpretation**: Business implications, assumptions, and next steps\n",
     "\n",
     ":::\n"
    ]
@@ -374,7 +374,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
@@ -418,8 +418,7 @@
     "    ax.legend()\n",
     "    ax.grid(alpha=0.3)\n",
     "\n",
-    "plt.tight_layout()\n",
-    "plt.show()"
+    "plt.tight_layout()"
    ]
   },
   {
@@ -619,7 +618,7 @@
     }
    ],
    "source": [
-    "df_transformed.plot()"
+    "df_transformed.plot();"
    ]
   },
   {
@@ -633,7 +632,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
@@ -816,7 +815,7 @@
     "print(f\"Std: {df['sales'].std():.2f}\")\n",
     "print(f\"Min: {df['sales'].min():.2f}\")\n",
     "print(f\"Max: {df['sales'].max():.2f}\")\n",
-    "print(\"\\nFirst few rows:\")\n",
+    "\n",
     "df.head()"
    ]
   },
@@ -925,9 +924,29 @@
     "\n",
     "We'll fit a model that includes a time trend, monthly seasonality, and week-of-month effects to capture the baseline sales pattern.\n",
     "\n",
+    "#### Understanding the Model Formula\n",
+    "\n",
+    "The formula `\"sales ~ 1 + t + C(month) + C(week_of_month)\"` specifies a linear regression model with the following components:\n",
+    "\n",
+    "- **`1`**: Intercept term (baseline level of sales)\n",
+    "- **`t`**: Linear time trend to capture long-term growth or decline\n",
+    "- **`C(month)`**: Categorical variable for month of year (1-12)\n",
+    "  - Creates 11 dummy variables (with one month as reference)\n",
+    "  - Captures annual seasonality patterns (e.g., higher sales in summer, lower in winter)\n",
+    "- **`C(week_of_month)`**: Categorical variable for week within month (1-5)\n",
+    "  - Creates 4 dummy variables (with week 1 as reference)\n",
+    "  - Captures within-month patterns (e.g., beginning vs end of month effects)\n",
+    "\n",
+    "This modeling approach is deliberately **simple and interpretable**. We're using categorical variables rather than smooth functions (like splines or Fourier terms) because:\n",
+    "1. **Flexibility**: Each month and week can have its own level without constraining the pattern\n",
+    "2. **Robustness**: Less prone to overfitting than complex non-linear models\n",
+    "3. **Interpretability**: Easy to understand which time periods have higher/lower baseline sales\n",
+    "\n",
+    "The key assumption is that these baseline patterns (trend + seasonality) would have **continued unchanged** in the post-intervention period if the TV promo hadn't occurred. Any deviation from this pattern is attributed to the campaign.\n",
+    "\n",
     ":::{note}\n",
     "The `random_seed` keyword argument for the PyMC sampler is not necessary. We use it here so that the results are reproducible.\n",
-    ":::\n"
+    ":::"
    ]
   },
   {
@@ -994,17 +1013,12 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Visualize the Results\n",
-    "\n",
-    "The plot shows three key panels:\n",
-    "1. **Top**: Observed data (black) vs model fit and counterfactual prediction (blue)\n",
-    "2. **Middle**: The causal impact at each time point (difference between observed and counterfactual)\n",
-    "3. **Bottom**: Model coefficients with uncertainty\n"
+    "### Visualize the Results"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
@@ -1101,7 +1115,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
@@ -1150,7 +1164,6 @@
     "    \"Total Absolute Lift from TV Promo Campaign\", fontsize=14, fontweight=\"bold\"\n",
     ")\n",
     "plt.tight_layout()\n",
-    "plt.show()\n",
     "\n",
     "# Get summary statistics\n",
     "absolute_lift_summary = az.summary(absolute_lift, kind=\"stats\")\n",
@@ -1215,7 +1228,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
@@ -1276,7 +1289,6 @@
     ")\n",
     "ax.set_xlabel(\"Percentage Lift (%)\")\n",
     "plt.tight_layout()\n",
-    "plt.show()\n",
     "\n",
     "# Summary statistics\n",
     "percentage_lift_summary = az.summary(avg_percentage_lift, kind=\"stats\")\n",
@@ -1304,7 +1316,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
@@ -1378,7 +1390,6 @@
     ")\n",
     "ax.set_xlabel(\"ROI (%)\")\n",
     "plt.tight_layout()\n",
-    "plt.show()\n",
     "\n",
     "# Summary statistics\n",
     "roi_summary = az.summary(roi, kind=\"stats\")\n",
@@ -1475,7 +1486,7 @@
     "\n",
     "The primary purpose of running lift tests is to generate experimental evidence that can improve your Media Mix Models. Here's how to use these results:\n",
     "\n",
-    "#### 1. **Feed Results into MMM Calibration**\n",
+    "#### 1. Feed Results into MMM Calibration\n",
     "\n",
     "The mean and standard deviation of the absolute lift (extracted above) can be directly integrated into MMM frameworks like PyMC-Marketing using the `add_lift_test_measurements()` method. This:\n",
     "- Constrains the MMM's saturation curves to match observed experimental data\n",
@@ -1484,15 +1495,15 @@
     "\n",
     "See the [PyMC-Marketing lift test calibration documentation](https://www.pymc-marketing.io/en/latest/notebooks/mmm/mmm_lift_test.html) for implementation details.\n",
     "\n",
-    "#### 2. **Run Multiple Lift Tests**\n",
+    "#### 2. Run Multiple Lift Tests\n",
     "\n",
     "To get the most value from lift testing:\n",
     "- Test **different channels** to calibrate each saturation curve\n",
     "- Test at **different spend levels** to better define the curve shape\n",
     "- Run tests in **different time periods** to capture seasonal variations\n",
     "- Accumulate results over time to build a robust calibration dataset\n",
     "\n",
-    "#### 3. **Additional Extensions**\n",
+    "#### 3. Additional Extensions\n",
     "\n",
     "- **Incorporating Other Data**: Include additional predictors (weather, competitors, events) in the ITS model to improve counterfactual accuracy\n",
     "- **Time-Varying Effects**: Examine `result.post_impact` at individual time points to understand how effects decay over time\n",
@@ -1505,32 +1516,26 @@
    "source": [
     "## Conclusion\n",
     "\n",
-    "This notebook demonstrated how to **conduct lift tests using Interrupted Time Series analysis** when control groups are unavailable. The key takeaways are:\n",
+    "This notebook demonstrated how to conduct lift tests using Interrupted Time Series analysis when traditional control groups are unavailable. We showed how ITS can measure the causal impact of a national TV campaign, extracting the key statistical outputs (mean lift and standard deviation) that are essential for Media Mix Model calibration. Beyond these primary outputs, we also calculated percentage lift and ROI to provide immediate business insights for decision-makers.\n",
     "\n",
-    "### What We Accomplished\n",
+    "The importance of lift testing extends beyond measuring individual campaigns. Lift tests provide experimental evidence that validates whether your marketing actually works, separating signal from noise in your attribution models. When these experimental results are fed into MMM calibration procedures, they constrain the model's saturation curves to match real-world observations, dramatically improving parameter identifiability—especially for correlated channels where the model might otherwise struggle to distinguish their individual effects. This leads to more accurate attribution estimates and better-informed budget allocation decisions.\n",
     "\n",
-    "1. **Generated a Lift Test**: Used ITS to measure the causal impact of a national TV campaign\n",
-    "2. **Extracted Key Metrics**: Calculated absolute lift with full uncertainty quantification (mean and standard deviation)\n",
-    "3. **Prepared Outputs for MMM**: Showed how to format results for direct integration into Media Mix Models\n",
-    "4. **Evaluated Business Impact**: Calculated percentage lift and ROI for immediate decision-making\n",
-    "\n",
-    "### Why This Matters\n",
+    ":::{admonition} The Complete Lift Testing Workflow\n",
+    ":class: tip\n",
     "\n",
-    "Lift tests provide **experimental evidence** that:\n",
-    "- Validates whether your marketing actually works\n",
-    "- Calibrates MMM models to improve accuracy\n",
-    "- Informs budget allocation decisions with confidence intervals\n",
-    "- Separates signal from noise in your attribution\n",
+    "For a complete marketing analytics workflow, lift testing fits into a three-step process:\n",
     "\n",
-    "### The Lift Testing Workflow\n",
+    "1. **Run lift tests** (this notebook) → Generate experimental evidence using ITS to measure the true causal impact of campaigns\n",
+    "2. **Calibrate your MMM** → Feed lift test results into your Media Mix Model using methods like PyMC-Marketing's `add_lift_test_measurements()` to constrain model parameters\n",
+    "3. **Optimize budget** → Use the calibrated MMM to allocate marketing spend across channels with confidence that the model reflects real-world effectiveness\n",
     "\n",
-    "For a complete marketing analytics workflow:\n",
+    "This approach bridges the gap between experimental causal inference and predictive modeling, combining the rigor of randomized experiments with the practical needs of ongoing attribution and optimization.\n",
+    ":::\n",
     "\n",
-    "1. **Run lift tests** (this notebook) → Generate experimental evidence using ITS\n",
-    "2. **Calibrate your MMM** → Use lift test results to constrain model parameters\n",
-    "3. **Optimize budget** → Use the calibrated MMM to allocate spend across channels\n",
+    "The method shown here is particularly valuable for national-level campaigns where geographic controls don't exist, but it can be extended in several ways. Multiple lift tests can be accumulated over time—testing different channels, different spend levels, and different time periods—to build a robust calibration dataset. The ITS model itself can be enhanced by incorporating additional predictors like weather, competitor activity, or special events to improve the counterfactual's accuracy. For organizations with multiple products or sub-markets, hierarchical Bayesian models can pool information across units while still estimating unit-specific lift.\n",
     "\n",
-    "This approach bridges the gap between experimental causal inference and predictive modeling, giving you both accuracy and actionable insights.\n",
+    "Ultimately, this approach represents a practical solution to one of marketing's most challenging problems: measuring incrementality when traditional experimental designs aren't feasible. By combining the strengths of time series analysis with Bayesian uncertainty quantification, we can generate the experimental evidence needed to build better attribution models and make more confident investment decisions.\n",
+    ":::\n",
     "\n",
     "### References\n",
     "\n",