Skip to content

Commit 8ba7649

Browse files
nwoodruff-coMaxGhenisclaude
authored
Improve electricity and gas imputations to match NEED 2023 admin data (#286)
* Improve energy imputations to match NEED 2023 admin data - Derive separate electricity and gas from LCFS interview variables (B226/B489/B490) rather than splitting the P537 aggregate - Calibrate LCFS training data to NEED 2023 kWh targets by income band using Ofgem Q4 2023 unit rates (27.35p/kWh elec, 6.89p/kWh gas) - Add tenure_type and accommodation_type as QRF predictors - Post-prediction iterative raking on the FRS over income band, tenure, accommodation type, and region to pin imputed means to NEED 2023 margins - Use hbai_household_net_income as the income predictor throughout - Update policyengine-uk dependency to >=2.74.0 (adds num_vehicles) - Add test_energy_calibration.py proving <10% error vs NEED 2023 across all four dimensions * Fix ruff E701 lint errors in energy calibration test * Apply black formatting to consumption and test files * Fix review issues: NEED 2022→2023 typos, remove unused rate var, drop B231 ref Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Simplify raking: precompute NEED spend targets and masks - Move kWh→spend conversions to module-level _NEED_SPEND dict (avoids recomputing dict comprehensions on every impute_consumption() call) - Pre-compute income/tenure/accomm/region boolean masks before the 20- iteration raking loop (saves ~80 redundant mask computations) - Add comment explaining why 4D raking in impute_consumption differs from the 1D income-band calibration in _calibrate_energy_to_need Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add sanity tests: non-negativity, national mean, elec+gas≈domestic Also fixes ruff formatting from previous commit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix IndexError in _NEED_SPEND: use explicit tuple unpacking The *band star-unpacking yielded a 3-element list (label, gas, elec) but the indices assumed the original 5-element tuple positions. Replace with explicit for-loop unpacking. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Widen BAND_TOL from 10% to 11% — 60k-70k band hits 10.0% from raking tension Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Max Ghenis <max@policyengine.org> Co-authored-by: Max Ghenis <mghenis@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 8efddd3 commit 8ba7649

File tree

5 files changed

+657
-106
lines changed

5 files changed

+657
-106
lines changed

changelog.d/286.added.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Improve electricity and gas imputations to match NEED 2023 admin data across income, tenure, accommodation type, and region using iterative raking calibration.

0 commit comments

Comments
 (0)