Skip to content

Commit 2375ab1

Browse files
JeanVanDykJeanVanDyk
committed
Update lift test
1 parent 74d3e77 commit 2375ab1

File tree

1 file changed

+85
-17
lines changed

1 file changed

+85
-17
lines changed

‎docs/source/notebooks/its_lift_test.ipynb‎

Lines changed: 85 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,7 @@
106106
},
107107
{
108108
"cell_type": "code",
109-
"execution_count": 1,
109+
"execution_count": null,
110110
"metadata": {},
111111
"outputs": [
112112
{
@@ -163,7 +163,7 @@
163163
},
164164
{
165165
"cell_type": "code",
166-
"execution_count": 2,
166+
"execution_count": null,
167167
"metadata": {},
168168
"outputs": [
169169
{
@@ -190,9 +190,13 @@
190190
"\n",
191191
"# Define intervention start (beginning of TV promo campaign)\n",
192192
"intervention_start = date_range[n_weeks_pre]\n",
193+
"# Define intervention end (after 4-week promo campaign)\n",
194+
"promo_weeks = 4\n",
195+
"intervention_end = intervention_start + pd.Timedelta(weeks=promo_weeks)\n",
193196
"print(f\"Pre-intervention period: {date_range[0]} to {date_range[n_weeks_pre - 1]}\")\n",
194197
"print(f\"Intervention starts: {intervention_start}\")\n",
195-
"print(f\"Post-intervention period: {intervention_start} to {date_range[-1]}\")\n",
198+
"print(f\"Intervention ends: {intervention_end}\")\n",
199+
"print(f\"Post-intervention period: {intervention_end} to {date_range[-1]}\")\n",
196200
"print(f\"Total weeks: {n_total}\")"
197201
]
198202
},
@@ -207,7 +211,7 @@
207211
},
208212
{
209213
"cell_type": "code",
210-
"execution_count": 3,
214+
"execution_count": null,
211215
"metadata": {},
212216
"outputs": [
213217
{
@@ -345,7 +349,6 @@
345349
"# 2. TV Promo: zero pre-intervention, high spend for 4 weeks during intervention\n",
346350
"spend_data[\"tv_promo\"] = np.zeros(n_total)\n",
347351
"# High spend during first 4 weeks of intervention (weeks 104-107)\n",
348-
"promo_weeks = 4\n",
349352
"for i in range(promo_weeks):\n",
350353
" # Promo spend around £50,000/week (much higher than baseline)\n",
351354
" spend_data[\"tv_promo\"][n_weeks_pre + i] = 50_000 + rng.normal(0, 3_000)\n",
@@ -374,7 +377,7 @@
374377
},
375378
{
376379
"cell_type": "code",
377-
"execution_count": 4,
380+
"execution_count": null,
378381
"metadata": {},
379382
"outputs": [
380383
{
@@ -436,7 +439,7 @@
436439
},
437440
{
438441
"cell_type": "code",
439-
"execution_count": 5,
442+
"execution_count": null,
440443
"metadata": {},
441444
"outputs": [],
442445
"source": [
@@ -492,7 +495,7 @@
492495
},
493496
{
494497
"cell_type": "code",
495-
"execution_count": 6,
498+
"execution_count": null,
496499
"metadata": {},
497500
"outputs": [
498501
{
@@ -526,7 +529,7 @@
526529
},
527530
{
528531
"cell_type": "code",
529-
"execution_count": 7,
532+
"execution_count": null,
530533
"metadata": {},
531534
"outputs": [
532535
{
@@ -722,7 +725,7 @@
722725
},
723726
{
724727
"cell_type": "code",
725-
"execution_count": 8,
728+
"execution_count": null,
726729
"metadata": {
727730
"tags": [
728731
"hide-input"
@@ -845,7 +848,7 @@
845848
},
846849
{
847850
"cell_type": "code",
848-
"execution_count": 9,
851+
"execution_count": null,
849852
"metadata": {
850853
"tags": [
851854
"hide-output"
@@ -902,6 +905,7 @@
902905
"result = cp.InterruptedTimeSeries(\n",
903906
" df,\n",
904907
" treatment_time=intervention_start,\n",
908+
" treatment_end_time=intervention_end,\n",
905909
" formula=\"sales ~ 1 + t + C(month) + C(week_of_month)\",\n",
906910
" model=cp.pymc_models.LinearRegression(sample_kwargs={\"random_seed\": seed}),\n",
907911
")"
@@ -911,12 +915,18 @@
911915
"cell_type": "markdown",
912916
"metadata": {},
913917
"source": [
914-
"### Visualize the Results"
918+
"### Visualize the Results\n",
919+
"\n",
920+
"The plot now shows the three-period design with:\n",
921+
"- **Solid red line**: Intervention start (when the TV promo campaign began)\n",
922+
"- **Dashed orange line**: Intervention end (when the 4-week campaign ended)\n",
923+
"\n",
924+
"This allows us to visually distinguish between the intervention period (when the campaign was active) and the post-intervention period (after the campaign ended, showing any persistence effects from adstock)."
915925
]
916926
},
917927
{
918928
"cell_type": "code",
919-
"execution_count": 10,
929+
"execution_count": null,
920930
"metadata": {},
921931
"outputs": [
922932
{
@@ -948,7 +958,7 @@
948958
},
949959
{
950960
"cell_type": "code",
951-
"execution_count": 11,
961+
"execution_count": null,
952962
"metadata": {},
953963
"outputs": [
954964
{
@@ -983,6 +993,64 @@
983993
"result.summary()"
984994
]
985995
},
996+
{
997+
"cell_type": "markdown",
998+
"metadata": {},
999+
"source": [
1000+
"### Period-Specific Effect Summaries\n",
1001+
"\n",
1002+
"With the three-period design, we can now analyze effects separately for the intervention period (when the campaign was active) and the post-intervention period (after the campaign ended). This helps us understand both the immediate impact and any persistence effects.\n"
1003+
]
1004+
},
1005+
{
1006+
"cell_type": "code",
1007+
"execution_count": null,
1008+
"metadata": {},
1009+
"outputs": [],
1010+
"source": [
1011+
"# Intervention period (during the 4-week campaign)\n",
1012+
"intervention_summary = result.effect_summary(period=\"intervention\")\n",
1013+
"print(\"Intervention Period Effect:\")\n",
1014+
"print(intervention_summary.text)\n",
1015+
"print(\"\\n\" + \"=\" * 60 + \"\\n\")\n",
1016+
"\n",
1017+
"# Post-intervention period (after campaign ended)\n",
1018+
"post_summary = result.effect_summary(period=\"post\")\n",
1019+
"print(\"Post-Intervention Period Effect:\")\n",
1020+
"print(post_summary.text)\n",
1021+
"print(\"\\n\" + \"=\" * 60 + \"\\n\")\n",
1022+
"\n",
1023+
"# Comparison summary (shows persistence metrics)\n",
1024+
"comparison_summary = result.effect_summary(period=\"comparison\")\n",
1025+
"print(\"Effect Persistence Analysis:\")\n",
1026+
"print(comparison_summary.text)"
1027+
]
1028+
},
1029+
{
1030+
"cell_type": "markdown",
1031+
"metadata": {},
1032+
"source": [
1033+
"### Detailed Persistence Analysis\n",
1034+
"\n",
1035+
"The `analyze_persistence()` method provides a comprehensive summary of how effects persist after the campaign ends:\n"
1036+
]
1037+
},
1038+
{
1039+
"cell_type": "code",
1040+
"execution_count": null,
1041+
"metadata": {},
1042+
"outputs": [],
1043+
"source": [
1044+
"persistence = result.analyze_persistence()\n",
1045+
"\n",
1046+
"# Access results programmatically if needed\n",
1047+
"print(\n",
1048+
" f\"\\nPersistence ratio: {persistence['persistence_ratio']:.3f} ({persistence['persistence_ratio'] * 100:.1f}%)\"\n",
1049+
")\n",
1050+
"print(f\"Mean effect during campaign: {persistence['mean_effect_during']:.2f}\")\n",
1051+
"print(f\"Mean effect post-campaign: {persistence['mean_effect_post']:.2f}\")"
1052+
]
1053+
},
9861054
{
9871055
"cell_type": "markdown",
9881056
"metadata": {},
@@ -1013,7 +1081,7 @@
10131081
},
10141082
{
10151083
"cell_type": "code",
1016-
"execution_count": 12,
1084+
"execution_count": null,
10171085
"metadata": {
10181086
"tags": [
10191087
"hide-input"
@@ -1181,7 +1249,7 @@
11811249
},
11821250
{
11831251
"cell_type": "code",
1184-
"execution_count": 13,
1252+
"execution_count": null,
11851253
"metadata": {
11861254
"tags": [
11871255
"hide-input"
@@ -1273,7 +1341,7 @@
12731341
},
12741342
{
12751343
"cell_type": "code",
1276-
"execution_count": 14,
1344+
"execution_count": null,
12771345
"metadata": {
12781346
"tags": [
12791347
"hide-input"

0 commit comments

Comments
 (0)