Skip to content

BOT: Fix #193: Add sd_sample() and iqr_sample() sharpness metrics#1103

Draft
nikosbosse wants to merge 1 commit intomainfrom
fix/193-sharpness-metrics-sample
Draft

BOT: Fix #193: Add sd_sample() and iqr_sample() sharpness metrics#1103
nikosbosse wants to merge 1 commit intomainfrom
fix/193-sharpness-metrics-sample

Conversation

@nikosbosse
Copy link
Collaborator

Summary

  • Adds sd_sample() and iqr_sample() as new sharpness metrics for sample-based forecasts
  • Both follow the existing mad_sample() pattern: observation-independent, row-wise apply() over the predicted matrix
  • Registered as default metrics in get_metrics.forecast_sample() under names "sd" and "iqr"

Fixes #193

Root cause

Only mad_sample() existed as a dedicated sharpness metric. The issue requested standard deviation and IQR as additional spread measures.

What the fix does

  • Implements sd_sample(observed = NULL, predicted, ...) using apply(predicted, 1, sd)
  • Implements iqr_sample(observed = NULL, predicted, ...) using apply(predicted, 1, IQR)
  • Adds both to the default metric list in get_metrics.forecast_sample()
  • Full roxygen documentation with examples and @export

Test coverage added

  • 11 new test blocks covering:
    • Correctness against known values (constant row, sequential row, mixed row)
    • observed = NULL handling
    • Error on missing predicted
    • Single-row edge case
    • Default metric registration
    • End-to-end score() integration
    • Agreement with reference apply()-based computation

Test plan

  • New tests fail before implementation
  • New tests pass after implementation (64/64)
  • Full test suite passes (706/706)
  • R CMD check: 0 errors, 0 warnings, 2 notes (pre-existing)

🤖 Generated with Claude Code

Add standard deviation and interquartile range as new sharpness metrics
for sample-based forecasts, following the existing mad_sample() pattern.
Both are observation-independent and included in the default metric list
for forecast_sample objects.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov
Copy link

codecov bot commented Feb 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.84%. Comparing base (ac0c01a) to head (0a57905).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1103   +/-   ##
=======================================
  Coverage   97.83%   97.84%           
=======================================
  Files          35       35           
  Lines        1845     1853    +8     
=======================================
+ Hits         1805     1813    +8     
  Misses         40       40           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Collaborator Author

@nikosbosse nikosbosse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLAUDE: Automated review — APPROVE. Clean, minimal implementation of sd_sample() and iqr_sample() that correctly follows the established mad_sample() pattern. All 11 test specs implemented and passing. No issues found — functions, docs, exports, imports, and metric registration are all correct.

@nikosbosse nikosbosse marked this pull request as draft February 13, 2026 08:28
@nikosbosse nikosbosse changed the title Fix #193: Add sd_sample() and iqr_sample() sharpness metrics BOT: Fix #193: Add sd_sample() and iqr_sample() sharpness metrics Feb 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add more metrics to assess sharpness of forecasts in a sample format

1 participant