Orange Telecom | Churn Intervention Program Model: Random Forest | PR-AUC: 0.906 | Test cohort: 667 customers
Each strategy below specifies the trigger condition, the intervention, the delivery owner by role, the acceptance criteria that defines success, the A/B test design, the KPIs, the timeline, and the infrastructure dependencies. Strategies 1 and 3 are batch-mode and activate in Phase 1. Strategy 2 requires real-time event streaming and activates in Phase 2, with a batch interim approach in Phase 1.
Trigger: Customer has active international plan AND churn probability >= 0.24 Action: Proactive plan audit call, offer plan optimization or loyalty credit Target segment: International Plan Holders (65 customers in test cohort, ~8% of base) Delivery owner: Customer Operations Director
Evidence:
- Segment churn rate: 38.5% vs 14.5% overall baseline
- Lift vs baseline: 2.64x
- Avg predicted probability: 35.9%
- Two root causes: (a) customers who use the plan heavily and feel overcharged, (b) customers who rarely use international calling but pay for the plan monthly
Expected impact:
- Reducing churn in this segment by 30% prevents ~12 churners per 1,000 customers/month
- At $500 LTV: $6,000/month in prevented revenue loss per 1,000 customers
Intervention design:
- For high-usage international customers: offer loyalty pricing or rate reduction
- For underusers (under 5 intl min/month): offer plan downgrade with loyalty credit This framing is perceived as transparent, trust-building, and reduces price-driven dissatisfaction
Trade-off:
- Revenue cannibalization from plan downgrades (~$5-15/month per affected customer)
- Mitigation: pair downgrade with 6-month contract extension or loyalty credit equal to one month's plan cost
Acceptance criteria: 90-day churn rate in the treated arm falls below 28% (vs. 38.5% observed baseline). Measured at 90 days post-launch with a minimum of 50 treated customers in the cohort. If fewer than 50 treated customers are available at launch, extend the measurement window to 120 days before drawing conclusions.
A/B test design:
- Split: 70% treated (proactive call), 30% control (no outreach)
- Eligibility: international plan holders with churn score >= 0.24
- Minimum detectable effect: 10 percentage point reduction in 90-day churn rate
- Required sample for 80% power at alpha=0.05: ~130 treated customers
- At ~8% of base, a 1,500+ customer operational base is required for 90-day significance, or a 3-4 month accumulation window at smaller scale
- Randomization unit: individual customer; randomize at score delivery time, not at call time
KPIs:
- Intervention rate: % of triggered customers contacted within 3 business days
- Contact-to-save rate: % of contacted customers with no cancellation event in 30 days
- 90-day churn rate: treated arm vs. control arm
- Revenue recovered: prevented churners x $500 LTV
- Cannibalization rate: plan downgrades as % of total conversions
Timeline: Phase 1, Week 6 (2 weeks after batch scoring validated at Phase 1 gate)
Infrastructure dependencies:
- CRM ingesting churn score and segment flag from daily batch job
- Agent desktop surfacing: customer's churn score, segment classification, intervention script for each sub-segment (high-usage vs. underuser)
Trigger: Customer reaches 2nd service call in a rolling 30-day window Action: Proactive outreach (call or push notification) before the 3rd contact Target segment: High Service Callers, 3+ calls (145 customers in test cohort, ~22% of base) Delivery owners: CRM Platform Lead (trigger infrastructure), Customer Operations Director (agent workflow)
Evidence:
- Segment churn rate: 31.7% vs 14.5% baseline
- Lift vs baseline: 2.18x
- The critical insight: each service contact is a customer expressing an unresolved problem. By the 3rd unresolved contact, the customer has mentally started evaluating alternatives. Intervening before the 3rd contact captures the window where frustration is high but commitment to leave is not yet formed.
Expected impact:
- Reducing 3+ service call churn by 25% prevents ~17 churners per 1,000 customers/month
- At $500 LTV: $8,500/month per 1,000 customers at moderate capture rate
Intervention design:
- After 2nd contact: trigger shifts customer to a senior retention specialist
- Offer: free technical support session, plan review, or temporary credit
- Key metric to track: rate of customers who contact a 3rd time after intervention
Trade-off:
- This strategy requires real-time service call event streaming, not batch scoring. Batch models score customers overnight, by the time they are flagged, the 3rd call may have already happened.
- Batch interim (Phase 1, Week 8): score customers with 2 calls in last 30 days each morning; retention specialist contacts flagged customers at start of business day. Not ideal, but it catches the recovery window for customers who have not yet made their 3rd call.
Acceptance criteria: Rate of customers making a 3rd service contact within 30 days after their 2nd contact falls below 40% in the treated group vs. the control group. 90-day churn rate in the treated group falls below 22% (vs. 31.7% observed baseline). Minimum 100 treated customers for statistical validity.
A/B test design:
- Split: 60% treated (proactive outreach trigger), 40% control (standard support flow)
- Eligibility: all customers reaching their 2nd service call in a 30-day window
- Randomization unit: customer; randomization must happen at the event-trigger moment, not in batch post-hoc, randomizing in batch allows the 3rd call to occur in the interim, contaminating the control group
- Batch interim version: randomize in morning batch based on previous day's call records; accept that some customers will have already made their 3rd call overnight
- Primary metric: 3rd contact rate in treated vs. control (leading indicator, faster to accrue)
- Secondary metric: 90-day churn rate (treated vs. control)
KPIs:
- Trigger latency: time from 2nd call event to agent notification (target: under 5 minutes in Phase 2 real-time; same morning in Phase 1 batch interim)
- 3rd contact rate: treated vs. control within 30 days of 2nd contact
- 90-day churn rate: treated vs. control
- Agent utilization rate: % of triggered customers contacted within the same business day
Timeline:
- Batch interim: Phase 1, Week 8
- Real-time activation: Phase 2, Week 14
Infrastructure dependencies:
- Phase 1 batch interim: CRM exports daily service call counts by customer; batch job flags 2-contact customers; agent desktop shows morning outreach list
- Phase 2 real-time:
- Event stream from service call system emitting structured event within 30s of call completion
- Inference API endpoint (sub-500ms p99 latency)
- Alert delivery to agent desktop within 5 minutes of trigger
- Event deduplication: same customer's 2nd call must not re-trigger within 24 hours
Trigger: Customer in top quartile of daytime usage (over 220 min/month in test data) AND no current unlimited day plan AND churn probability >= 0.24 Action: Proactively offer unlimited or capped-rate day plan with a $5-10/month incentive Target segment: Heavy Daytime Users, top 25% (167 customers in test cohort, ~25% of base) Delivery owner: Customer Operations Director
Evidence:
- Segment churn rate: 27.5% vs 14.5% baseline
- Lift vs baseline: 1.89x
- Avg predicted probability: 27.2%
- Daytime usage is the top-ranked feature by both Random Forest importance and SHAP values. Heavy users face compounding per-minute billing exposure each month, a structural pricing friction point that persists regardless of support quality.
Expected impact:
- Converting 35% of flagged segment to a plan that removes billing friction reduces churn rate in that segment by an estimated 20-30%
- Net revenue effect: plan upgrade reduces per-minute revenue but increases contract tenure and NPS, LTV increase outpaces short-term ARPU loss
Intervention design:
- Batch model scoring: flag top-quartile users monthly
- Offer via app push or SMS: "Your usage qualifies you for our Unlimited Day plan, $X/month"
- Require 2 or more consecutive months of top-quartile usage before offer (prevents one-time spikes)
Trade-off:
- Revenue cannibalization if loyal customers who would not churn self-select into cheaper plans
- Mitigation: only offer to customers with churn probability at or above threshold (0.24), not to all heavy users
Acceptance criteria: Plan conversion rate in the treated group reaches or exceeds 25% (offer accepted, not just sent). 90-day churn rate in customers who accept the plan falls below 15% (vs. 27.5% segment baseline). Net revenue impact is positive at 6 months, accounting for plan revenue delta on converted customers vs. $500 LTV value of prevented cancellations.
A/B test design:
- Split: 70% treated (plan upgrade offer via SMS/app), 30% control (no offer)
- Eligibility: top-quartile daytime users with churn score >= 0.24, not currently on unlimited plan, with 2+ consecutive months at top-quartile usage
- Key confound: customers who accept the offer self-select into treatment. Use intent-to-treat analysis (offer sent, not offer accepted) as the primary metric to avoid self-selection bias. Track offer acceptance rate as a secondary metric.
- Randomization unit: customer; randomize at offer generation time in monthly batch
KPIs:
- Offer acceptance rate: % of offer-sent customers who convert to new plan
- Plan migration rate: net plan changes (upgrades minus downgrades) in treated segment
- 90-day churn rate: treated vs. control, intent-to-treat primary analysis
- ARPU delta: plan revenue change per converted customer (net of per-minute revenue loss)
- Net LTV change at 6 months: (churners prevented x $500) minus (ARPU loss x customer count)
Timeline: Phase 1, Week 7 (requires SMS/app push capability confirmed at Phase 0 gate)
Infrastructure dependencies:
- SMS or app push notification capability with delivery confirmation
- CRM plan type field accessible for eligibility check
- Monthly batch scoring with usage history lookback (2+ months required)
- Finance review of ARPU delta estimates before launch
| Strategy | Segment Churn Rate | Lift | Est. Cost/Churner Prevented | Execution Complexity | Priority |
|---|---|---|---|---|---|
| International Plan Audit | 38.5% | 2.64x | ~$50/intervention | Low (batch) | 1 |
| Service Contact Intervention | 31.7% | 2.18x | ~$20/call | High (real-time) | 2 |
| Usage-Based Plan Upgrade | 27.5% | 1.89x | ~$30/offer | Low (batch) | 3 |
Recommended sequencing: Strategies 1 and 3 activate in Phase 1 as batch monthly campaigns, no new infrastructure and testable within 60 days of Phase 1 launch. Strategy 2 activates in Phase 2 as a real-time trigger system over 6 months. A/B test each strategy against a control group before scaling. Do not combine all three into a single test, independent A/B tests allow per-strategy ROI attribution.
If all three strategies achieve their target capture rates:
- Strategies 1 and 3 (batch): ~25 churners prevented per 1,000 customers/month
- Strategy 2 (real-time, Phase 2): ~17 additional churners prevented per 1,000/month
- Total: up to 42 churners prevented per 1,000 customers/month
- At $500 LTV: $21,000/month in prevented revenue loss per 1,000 customers
These are model-derived estimates. A/B test validation is required before treating these as realized savings. Full delivery timeline and risk register: program_delivery_plan.md