Skip to content

feat: add multi-country forecast evaluation pipeline#141

Open
SGHoekstra wants to merge 2 commits intobancaditalia:mainfrom
SGHoekstra:feat/multicountry-pipeline-v2
Open

feat: add multi-country forecast evaluation pipeline#141
SGHoekstra wants to merge 2 commits intobancaditalia:mainfrom
SGHoekstra:feat/multicountry-pipeline-v2

Conversation

@SGHoekstra
Copy link
Collaborator

Summary

Adds utilities and example scripts for evaluating ABM forecast performance across countries and model variants (base, CANVAS, GrowthRate AR(1)).

  • src/utils/bias_ttest.jl — one-sample HAC t-test (Newey-West SE) for forecast bias significance
  • src/utils/analysis_utils.jl — forecast error aggregation, country discovery, CSV/LaTeX table writers
  • src/utils/error_tables.jl — per-country RMSE/bias tables with AR and VAR benchmarks and Diebold-Mariano tests
  • src/utils/save_all_predictions.jl — adds model_factory kwarg and run_variant_pipeline convenience wrapper
  • examples/analysis/tabs/simulate_and_create_tables_multicountry.jl — runs the full pipeline for all calibrated countries
  • examples/analysis/tabs/create_tables.jl — single-country table generation
  • examples/analysis/tabs/aggregate_forecast_results.jl — aggregates per-country CSVs into cross-country summaries
  • examples/analysis/figs/aggregate_forecast_results.jl — heatmaps of forecast performance across countries

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant