Reduce peak memory in demand flex to fix OOM on ConEd/Nimo runs 15-16

[alxsmith]

What

The demand-flexibility pipeline (utils/cairo.py apply_runtime_tou_demand_response + utils/demand_flex.py) OOMs (exit code 137) on runs 15-16 for ConEd (~15,435 buildings) and Nimo (~6,791 buildings) on a 30 GB instance. These runs apply demand flex to all buildings (not just HP), so the full load DataFrame goes through the shift pipeline.

Why

Runs 15-16 are required for the complete 16-run NY batch. Without them, we can't produce default-structure demand-flex bills for ConEd and Nimo. Currently blocked — other 5 NY utilities complete fine because they have fewer buildings.

How

Peak memory is ~18-20 GB for load data alone because several full copies coexist:

1. raw_load_elec (~4 GB)
2. tou_df — full-year copy of TOU buildings (~5 GB)
3. Per-season shifted results accumulated in shifted_chunks list (~5 GB per season)
4. shifted_load_elec = raw_load_elec.copy() (~4 GB)

Proposed fixes (Option 1 validated as bit-identical to current output):

1. Vectorize process_residential_hourly_demand_response_shift — replace the per-building groupby loop + pd.concat with a single merge + vectorized column ops. Eliminates ~3-5 GB of fragmented DataFrame overhead. Already implemented on branch 371-add-default-structure-tariff-generation-from-monthly_rates-yaml. ~15x speedup.
2. Eliminate tou_df — build season slices directly from raw_load_elec instead of creating a full-year intermediate copy. Saves ~5 GB.
3. Write shifted values back per-season — instead of accumulating shifted_chunks across all seasons, allocate shifted_load_elec upfront and write each season's results immediately. Saves ~2-5 GB.

Combined, these bring peak from ~18 GB to ~8-10 GB, fitting comfortably in 30 GB.

Deliverables

• PR implementing Options 2 and 3 in utils/cairo.py (Option 1 already done on the default-tariff branch)
• del of Phase 1a intermediates in utils/demand_flex.py (already done on the default-tariff branch)
• ConEd and Nimo runs 15-16 complete without OOM on 30 GB instance
• No change to numerical results (verified by test comparing pre/post outputs)

[linear]

RDP-176