Skip to content

Fix stable totals for age-filtered clinical trial search#195

Merged
imaurer merged 1 commit intomainfrom
fix-age-filter-pagination
Mar 11, 2026
Merged

Fix stable totals for age-filtered clinical trial search#195
imaurer merged 1 commit intomainfrom
fix-age-filter-pagination

Conversation

@imaurer
Copy link
Collaborator

@imaurer imaurer commented Mar 11, 2026

Summary

  • Age-filtered trial searches previously reported inconsistent totals: the count was derived from a prefix of the result set, so --count-only output changed with --limit
  • Add count_all which traverses all CTGov pages (1000 results/page) and applies age verification across the complete population, returning the definitive verified count
  • Route --count-only through count_all so count-only calls always return the true verified total (e.g. Total: 472 for melanoma/recruiting/age-51, stable across any --limit)
  • Paginated search_page calls now return total=None when traversal does not exhaust all pages, surfacing "total unknown" in the CLI footer instead of a misleading lower bound
  • Expose ClinicalTrialsClient::new_for_test under #[cfg(test)] for unit testing
  • Add age-filter count stability spec assertions to spec/04-trial.md

Test plan

  • cargo test passes (574 tests)
  • make spec passes (96 passed, 5 skipped)
  • biomcp search trial -c melanoma -s recruiting --age 51 --count-only returns Total: 472
  • Same command with --limit 10, --limit 20, --limit 50 all return Total: 472
  • biomcp search trial -c melanoma -s recruiting --age 51 --limit 10 shows total unknown footer
  • biomcp search trial -c melanoma -s recruiting --age 51 --limit 10 --offset 20 shows paginated rows with total unknown

Age-filtered trial searches reported inconsistent totals because the
count was derived from a prefix of the result set rather than the full
verified population. The reported total changed with --limit.

- Add `count_all` which pages through all CTGov results with a 1000-row
  page size and applies age verification across the complete set, then
  returns the definitive verified count
- Route `--count-only` through `count_all` instead of `search_page`,
  so count-only calls always return the true verified total
- Paginated (`search_page`) age-filtered calls now return `total=None`
  when traversal does not exhaust all pages, surfacing "total unknown"
  in the CLI footer instead of a misleading lower bound
- Expose `ClinicalTrialsClient::new_for_test` under `#[cfg(test)]`
- Add age-filter count stability spec section to `spec/04-trial.md`
@imaurer imaurer merged commit ff4fde1 into main Mar 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant