Skip to content

Conversation

@amrit110
Copy link
Member

This pull request introduces a monthly error analysis feature to the evaluation dashboard and adds supporting backend and utility scripts for improved forecasting evaluation. The main changes include a new backend API endpoint for monthly metrics, frontend integration with visualization and error highlighting, and utility scripts to support unbiased evaluation data management.

Monthly Error Analysis Feature

  • Added a new backend API endpoint /evaluation/static/monthly to provide monthly RMSE/MAE error metrics, enabling the identification of seasonal patterns and months with highest/lowest errors.
  • Integrated monthly analysis into the frontend: fetches data from the new endpoint, displays a summary of best/worst months, trend analysis, and a line chart of monthly errors in EvaluationPage. [1] [2] [3]
  • Defined new TypeScript interfaces (MonthlyMetrics, MonthlyAnalysis) to type the monthly analysis data structure.
  • Updated chart imports to support new visualizations.

Evaluation Data Management Utilities

  • Added scripts/clean_dynamic_eval.py, a utility script to safely delete dynamic evaluation data from BigQuery, with interactive and force modes, to prepare for unbiased (hourly) evaluation.
  • Updated scripts/populate_dynamic_evaluation.py to default to hourly (--interval 1) predictions for unbiased evaluation, and added a warning if a larger interval is used, to prevent temporal bias in metrics. [1] [2]

@amrit110 amrit110 self-assigned this Dec 10, 2025
@amrit110 amrit110 added bug Something isn't working enhancement New feature or request labels Dec 10, 2025
@amrit110 amrit110 merged commit 4068fb8 into main Dec 10, 2025
10 checks passed
@amrit110 amrit110 deleted the add_errors_section branch December 10, 2025 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants