Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -171,13 +171,16 @@ The column shows: *(still on track) / (total eligible) = percentage*
---

#### Revisit Dist.
**What it shows:** The average distance (in kilometers) between successive GPS coordinates when the FLW revisits the **same mother**.
**What it shows:** The average distance (in kilometers) between successive GPS coordinates when the FLW revisits the **same mother**, with a denominator showing how many cases contributed.

**How it's calculated:**
1. Group all visits by mother (using the mother case ID)
2. Sort each mother's visits by date/time
3. For each pair of consecutive visits to the same mother, calculate the straight-line distance between GPS coordinates using the Haversine formula (accounts for Earth's curvature)
4. Report the **average** of all these distances for the FLW
5. Display a denominator "(N)" where N is the number of mother cases that had 2 or more GPS-tagged visits — i.e., the cases that could actually be compared

**Example:** "0.3 km (12)" means 12 mothers had repeat visits and the average revisit distance was 0.3 km.

**Why this matters:** When an FLW visits the same mother multiple times, the GPS coordinates should be close together (same household). Large distances between revisits to the same mother suggest the FLW may not be visiting the actual location.

Expand Down Expand Up @@ -233,6 +236,15 @@ The column shows: *(still on track) / (total eligible) = percentage*

---

#### Dist. Ratio
**What it shows:** A ratio comparing revisit distance to inter-visit travel distance.

**How it's calculated:** (Revisit Dist. in meters) / (Meter/Visit in meters). Equivalently: Revisit Dist. (km) x 1000 / Meter/Visit (m). A high ratio means the FLW's revisits to the same mother are spread far apart relative to how far they travel between different mothers — which may indicate GPS anomalies.

**Why this matters:** Revisit Dist. and Meter/Visit each tell part of the story. The ratio combines them into a single signal: an FLW who travels short distances between different mothers (low Meter/Visit) but has large revisit distances (high Revisit Dist.) will have a high Dist. Ratio, flagging a potential concern.

---

#### Phone Dup %
**What it shows:** The percentage of the FLW's mothers whose phone numbers appear more than once across the FLW's caseload.

Expand Down Expand Up @@ -363,8 +375,14 @@ All GPS Analysis columns derive from these form fields:
- **App build version:** `form.meta.app_build_version` (integer, used for optional version filtering)
- **Form name:** `form.@name` (identifies visit type)

### Aggregate Map

At the top of the GPS tab, a collapsible map displays all FLW visits on a single view. Each FLW's visits are shown as color-coded pins (a unique color per FLW), so you can visually compare coverage areas and spot overlapping or isolated clusters. The map uses marker clustering — when zoomed out, nearby pins are grouped into numbered clusters that expand as you zoom in. This keeps the map responsive even with thousands of visits. The map is **collapsed by default**; click the toggle to expand it.

### FLW Table Columns

All columns in the GPS table are **sortable** — click any column header to sort ascending or descending. This makes it easy to find outliers (e.g., sort by Dist. Ratio descending to surface the most suspicious FLWs first).

#### Total Visits
The number of form submissions by this FLW within the date range.

Expand All @@ -391,20 +409,34 @@ The number of distinct mother cases visited by this FLW within the date range.

---

#### Avg Case Dist
**What it shows:** Average distance (km) between consecutive visits to the same mother.
#### Revisit Dist.
**What it shows:** Average distance (km) between consecutive visits to the same mother, with a denominator showing how many cases contributed to the calculation.

**How it's calculated:** Same as Revisit Dist. in the Overview tab — average Haversine distance between consecutive GPS coordinates for visits to the same mother case, across all mothers for this FLW.
**How it's calculated:** Same as Revisit Dist. in the Overview tab — average Haversine distance between consecutive GPS coordinates for visits to the same mother case, across all mothers for this FLW. Displayed as a value followed by "(N)" where N is the number of mother cases that had 2 or more GPS-tagged visits (i.e., the number of cases that could be compared). For example, "0.3 km (12)" means 12 mothers had repeat visits and the average revisit distance was 0.3 km.

---

#### Max Case Dist
#### Max Revisit Dist.
**What it shows:** The single largest distance (km) observed between consecutive visits to the same mother.

**Color coding:** Red and bold when exceeding 5 km.

---

#### Meter/Visit
**What it shows:** The median distance (in meters) the FLW travels between consecutive visits to **different mothers within a single day**. Same calculation as the Meter/Visit column in the Overview tab.

**Red flag:** Values below 100 meters are highlighted in red.

---

#### Dist. Ratio
**What it shows:** A ratio comparing revisit distance to inter-visit travel distance.

**How it's calculated:** (Revisit Dist. in meters) / (Meter/Visit in meters). In other words: Revisit Dist. (km) x 1000 / Meter/Visit (m). A high ratio means the FLW's revisits to the same mother are spread far apart relative to how far they travel between different mothers — which may indicate GPS anomalies.

---

#### Trailing 7 Days
**What it shows:** A sparkline bar chart showing the FLW's daily travel pattern over the last 7 days.

Expand All @@ -423,7 +455,7 @@ Clicking "Details" on an FLW row expands to show individual visit records with:
- **Form:** Type of visit (from `form.@name`)
- **Entity:** Mother case name (from `form.mbw_visit.deliver.entity_name`)
- **GPS:** Latitude and longitude coordinates (from `form.meta.location`)
- **Dist from Prev:** Distance from the previous visit to the same mother (computed via Haversine)
- **Revisit Dist.:** Distance from the previous visit to the same mother (computed via Haversine)
- **Status:** Whether the visit is flagged (distance > 5 km)

---
Expand Down Expand Up @@ -694,7 +726,7 @@ When creating a task for an FLW (via the OCS AI integration), the system automat
| Indicator | Threshold |
|-----------|-----------|
| Visit flagged | Distance from previous visit to same mother > 5 km |
| Max Case Dist highlighted | > 5 km |
| Max Revisit Dist. highlighted | > 5 km |
| Meter/Visit red flag | < 100 meters |

---
Expand Down
34 changes: 22 additions & 12 deletions commcare_connect/workflow/templates/mbw_monitoring/DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,7 @@ The stream view executes 7 steps, yielding progress messages at each stage:

Data is NOT sent as one large final payload. Instead, the backend sends 3 separate `data_section` SSE events to prevent OOM on large opportunities (50K+ visits):

1. **GPS section** — `gps_data` (FLW summaries without visits, date range, flag threshold)
1. **GPS section** — `gps_data` (FLW summaries without visits, date range, flag threshold, all_coordinates for aggregate map)
2. **Follow-up section** — `followup_data` (per-FLW/per-mother metrics, visit status distribution)
3. **Overview + Performance section** — `overview_data`, `performance`, monitoring session metadata

Expand Down Expand Up @@ -380,7 +380,8 @@ The final SSE payload (`data.data`) contains:
"total_flagged": 12,
"date_range_start": "2025-01-01",
"date_range_end": "2025-01-31",
"flw_summaries": [...]
"flw_summaries": [...], // Each summary includes cases_with_revisits and median_meters_per_visit
"all_coordinates": [...] // Array of {lat, lng, username, entity, date, flagged} for aggregate map
},
"followup_data": {
"total_cases": 300,
Expand Down Expand Up @@ -439,7 +440,7 @@ Provides a bird's-eye view of each FLW's performance by merging data from all so
- Interactive legend below the chart: click categories to toggle visibility on/off (dimmed + strike-through when hidden)
- Bar heights proportional to count (tallest bar = full height, others scaled)

**FLW Table Columns** (13 columns, toggleable via Column Selector):
**FLW Table Columns** (14 columns, toggleable via Column Selector):

| Column | Data Source | Description |
|--------|-----------|-------------|
Expand All @@ -449,9 +450,10 @@ Provides a bird's-eye view of each FLW's performance by merging data from all so
| Post-Test | TBD | Post-test attempts (placeholder, shows "--") |
| Follow-up Rate | Follow-up analysis | % of visits due 5+ days ago that are completed, among eligible mothers |
| Eligible 5+ | Drill-down data | Eligible mothers still on track (5+ completed OR <=1 missed). Color: green >=70%, yellow 50-69%, red <50% |
| Revisit Dist. | GPS analysis | Median haversine distance (km) between revisits to the same mother |
| Revisit Dist. | GPS analysis | Median haversine distance (km) between revisits to the same mother, with "(N)" denominator showing cases with 2+ GPS visits |
| Meter/Visit | GPS analysis | Median meters traveled per visit (configurable app version filter via Filter bar) |
| Minute/Visit | GPS analysis | Median minutes per visit |
| Dist. Ratio | GPS analysis | Revisit distance x 1000 / meter per visit. Higher values may indicate suspicious patterns |
| Phone Dup % | Quality metrics | % of mothers sharing duplicate phone numbers |
| ANC = PNC | Quality metrics | Count of mothers where ANC and PNC completion dates match |
| Parity | Quality metrics | Parity value concentration (% duplicate + mode) |
Expand All @@ -460,7 +462,7 @@ Provides a bird's-eye view of each FLW's performance by merging data from all so
| % EBF | Pipeline (bf_status) | % of FLW's postnatal visits reporting exclusive breastfeeding. Color: green 50-85%, yellow 31-49% or 86-95%, red 0-30% or 96-100%. Red flag in OCS prompt when in red zone. |
| Actions | Action handlers | Assessment buttons, notes, filter, task creation (locked, always visible) |

**Column Selector**: Dropdown next to "FLW Overview" title showing N/16 visible columns. Toggle individual columns, "Show All", or "Minimal" presets.
**Column Selector**: Dropdown next to "FLW Overview" title showing N/17 visible columns. Toggle individual columns, "Show All", or "Minimal" presets.

**Actions per FLW** (Overview tab only - other tabs have Filter only):

Expand All @@ -477,7 +479,7 @@ Identifies potential fraud or GPS anomalies by analyzing distances between conse

**Summary Cards**: Total Visits, Flagged Visits, Date Range, Flag Threshold (5 km)

**FLW Table Columns**:
**FLW Table Columns** (all columns are sortable):

| Column | Description |
|--------|-------------|
Expand All @@ -486,13 +488,17 @@ Identifies potential fraud or GPS anomalies by analyzing distances between conse
| With GPS | Count + percentage |
| Flagged | Visits exceeding 5km threshold (highlighted red) |
| Unique Cases | Distinct mother_case_id count |
| Avg Case Dist | Average distance between visits to same case (km) |
| Max Case Dist | Maximum distance (red if >5km) |
| Revisit Dist. | Median haversine distance (km) between revisits to the same mother, with "(N)" denominator showing cases with 2+ GPS visits |
| Max Revisit Dist. | Maximum revisit distance (red if >5km) |
| Meter/Visit | Median haversine distance (m) between consecutive visits to different mothers on the same day. Color-coded: green >=1000m, yellow >=100m, red <100m |
| Dist. Ratio | Revisit distance x 1000 / meter per visit. Higher values may indicate suspicious patterns (close revisits but far daily travel, or vice versa) |
| Trailing 7 Days | Sparkline bar chart of daily travel distance |

**Aggregate Map**: A collapsible map at the top of the GPS tab showing all FLW visits with color-coded pins (HSL hue rotation per FLW). Uses MarkerCluster for performance with large datasets. Each popup shows FLW name, entity, date, and flagged status. Collapsed by default — click to expand. A legend below the map shows the FLW-to-color mapping.

**Actions per FLW**: Filter button, Details drill-down button (no assessment or task buttons)

**Drill-Down**: Clicking "Details" on a FLW row expands an inline panel showing individual visit records with date, form name, entity, GPS coordinates, distance from previous visit, and flagged status.
**Drill-Down**: Clicking "Details" on a FLW row expands an inline panel showing individual visit records with date, form name, entity, GPS coordinates, revisit distance (haversine distance from previous visit to the same mother), and flagged status.

### Follow-Up Rate Tab

Expand Down Expand Up @@ -585,7 +591,7 @@ Used for: Dynamic xmlns discovery
Pipeline Visit Forms (Connect API)
|
+-- username ----------> FLW Names (Connect API)
+-- GPS coordinates --> GPS Analysis (Haversine, meter/visit, min/visit)
+-- GPS coordinates --> GPS Analysis (Haversine, meter/visit, min/visit, dist. ratio, aggregate map)
+-- form_name ---------> Visit type normalization (FORM_NAME_TO_VISIT_TYPE)
+-- mother_case_id ----> Mother-to-FLW mapping
+-- parity ------------> Quality metrics (from ANC Visit rows)
Expand Down Expand Up @@ -1047,15 +1053,19 @@ Django uses `CompressedManifestStaticFilesStorage` (whitenoise). The `{% static
| SSE streaming with progress | All | Real-time loading messages during data loading |
| FLW filter (multi-select) | All | Filter by FLW name across all tabs |
| Mother filter (multi-select) | Follow-Up | Filter by mother name |
| Column selector | Overview | Toggle 16 columns with Show All / Minimal presets |
| Column sorting | All | Click column headers to sort asc/desc |
| Column selector | Overview | Toggle 17 columns with Show All / Minimal presets |
| Column sorting | All | Click column headers to sort asc/desc (all GPS tab columns are now sortable) |
| Horizontal table scrolling | Overview | Scroll wrapper with `width: 0; minWidth: 100%` pattern |
| GPS drill-down | GPS | Click "Details" sets `expandedGps` state; `useEffect([expandedGps, dashData, instance.opportunity_id, appliedAppVersionOp, appliedAppVersionVal])` checks for embedded visits (snapshot) first, then lazy-loads from `/api/gps/<username>/` |
| Follow-up drill-down | Follow-Up | Per-mother visit details with metadata |
| Visit status distribution | Overview | Per-visit-type stacked bar chart (6 bars) with toggleable legend and "Not Due Yet" category |
| Per-visit-type breakdown | Follow-Up | ANC through Month 6 mini columns |
| Trailing 7-day sparkline | GPS | Daily travel distance bar chart |
| GPS flag threshold (5km) | GPS | Red highlighting for suspicious distances |
| Aggregate GPS map | GPS | Collapsible map at top of tab showing all FLW visits with color-coded pins (HSL hue rotation), MarkerCluster for performance, FLW legend below map |
| Meter/Visit column | GPS, Overview | Median haversine distance (m) between consecutive visits to different mothers on same day. Color-coded: green >=1000m, yellow >=100m, red <100m |
| Dist. Ratio column | GPS, Overview | Revisit distance x 1000 / meter per visit — higher values may indicate suspicious patterns |
| Revisit denominator | GPS, Overview | Revisit Dist. now shows "(N)" where N = cases with 2+ GPS visits |
| Follow-up rate (business def) | Follow-Up | Eligibility + grace period filtered rate |
| GS Score from CCHQ | Overview | First Gold Standard score from supervisor app |
| Quality/fraud metrics | Overview | Phone dup, parity/age concentration, ANC=PNC, age=reg |
Expand Down
17 changes: 17 additions & 0 deletions commcare_connect/workflow/templates/mbw_monitoring/gps_analysis.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ class FLWSummary:
unique_cases: int
avg_case_distance_km: float | None
max_case_distance_km: float | None
cases_with_revisits: int # mothers with >1 GPS visit (revisit distance denominator)
trailing_7_days: list[DailyTravel]
avg_daily_travel_km: float | None

Expand Down Expand Up @@ -393,6 +394,13 @@ def build_result_from_analyzed_visits(
total_km = sum(d.total_distance_km for d in trailing_7)
avg_daily_travel = total_km / len(trailing_7)

# Count mothers with >1 GPS visit (revisit distance denominator)
cases_with_revisits = len({
v.mother_case_id or v.case_id
for v in flw_visits
if v.distance_from_prev_case_visit is not None
})

flw_summaries.append(
FLWSummary(
username=username,
Expand All @@ -405,6 +413,7 @@ def build_result_from_analyzed_visits(
if case_distances
else None,
max_case_distance_km=meters_to_km(max(case_distances)) if case_distances else None,
cases_with_revisits=cases_with_revisits,
trailing_7_days=trailing_7,
avg_daily_travel_km=avg_daily_travel,
)
Expand Down Expand Up @@ -479,6 +488,13 @@ def analyze_gps_metrics(
total_km = sum(d.total_distance_km for d in trailing_7)
avg_daily_travel = total_km / len(trailing_7)

# Count mothers with >1 GPS visit (revisit distance denominator)
cases_with_revisits = len({
v.mother_case_id or v.case_id
for v in flw_visits
if v.distance_from_prev_case_visit is not None
})

flw_summaries.append(
FLWSummary(
username=username,
Expand All @@ -491,6 +507,7 @@ def analyze_gps_metrics(
if case_distances
else None,
max_case_distance_km=meters_to_km(max(case_distances)) if case_distances else None,
cases_with_revisits=cases_with_revisits,
trailing_7_days=trailing_7,
avg_daily_travel_km=avg_daily_travel,
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ def serialize_flw_summary(flw: FLWSummary) -> dict:
"unique_cases": flw.unique_cases,
"avg_case_distance_km": round(flw.avg_case_distance_km, 2) if flw.avg_case_distance_km else None,
"max_case_distance_km": round(flw.max_case_distance_km, 2) if flw.max_case_distance_km else None,
"cases_with_revisits": flw.cases_with_revisits,
"trailing_7_days": [serialize_daily_travel(dt) for dt in flw.trailing_7_days],
"avg_daily_travel_km": round(flw.avg_daily_travel_km, 2) if flw.avg_daily_travel_km else None,
}
Expand Down
Loading