Add sparse matrix builder for local area calibration - SNAP targets (#456)

* Add sparse matrix builder for local area calibration

Core components:
- sparse_matrix_builder.py: Database-driven approach for building calibration matrices
- calibration_utils.py: Shared utilities (cache clearing, constraints, geo helpers)
- matrix_tracer.py: Debugging utility for tracing through sparse matrices
- create_stratified_cps.py: Create stratified sample preserving high-income households
- test_sparse_matrix_builder.py: 6 verification tests for matrix correctness

Data pipeline changes:
- Add GEO_STACKING env var to cps.py and puf.py for geo-stacking data generation
- Add GEO_STACKING_MODE env var to extended_cps.py
- Add CPS_2024_Full, PUF_2023, ExtendedCPS_2023 classes
- Add policy_data.db download to prerequisites
- Add 'make data-geo' target for geo-stacking data pipeline

CI/CD:
- Add geo-stacking dataset build step to workflow
- Add sparse matrix builder test step after geo data generation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* Add changelog entry and format code

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* Refactor tests and fix enum encoding, minimize PR scope

- Move sparse matrix tests to tests/test_local_area_calibration/
- Split large test file into focused modules (column indexing, same-state,
  cross-state, geo masking)
- Fix small_enhanced_cps.py enum encoding (decode_to_str before astype)
- Fix create_stratified_cps.py to use local storage instead of HuggingFace
- Remove CPS_2024_Full to keep PR minimal
- Revert ExtendedCPS_2024 to use CPS_2024

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* Rename GEO_STACKING to LOCAL_AREA_CALIBRATION and restore tracer functionality

- Rename GEO_STACKING to LOCAL_AREA_CALIBRATION in cps.py, puf.py, extended_cps.py
- Rename data-geo to data-local-area in Makefile and workflow
- Add create_target_groups function to calibration_utils.py
- Enhance MatrixTracer with get_group_rows method and variable_desc in row catalog
- Add TARGET GROUPS section to print_matrix_structure output
- Add local_area_calibration_setup.ipynb documentation notebook

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* Clear notebook outputs for Myst compatibility

* Pin mystmd>=1.7.0 to fix notebook rendering in docs

* Add logging for constraint evaluation failures and document CD GEOID format

- Replace silent exception catch with debug logging for constraint evaluation
- Add comment explaining CD GEOID format (SSCCC where SS=state FIPS)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: Max Ghenis <[email protected]>

Author: 3 people

.github/workflows/reusable_test.yaml

@@ -75,6 +75,18 @@ jobs:
           TEST_LITE: ${{ !inputs.upload_data }}
           PYTHON_LOG_LEVEL: INFO
 
+      - name: Build datasets for local area calibration
+        if: inputs.full_suite
+        run: |
+          LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/cps/cps.py
+          LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/puf/puf.py
+          LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/cps/extended_cps.py
+          python policyengine_us_data/datasets/cps/local_area_calibration/create_stratified_cps.py 10500
+
+      - name: Run local area calibration tests
+        if: inputs.full_suite
+        run: pytest policyengine_us_data/tests/test_local_area_calibration/ -v
+
       - name: Save calibration log
         if: inputs.full_suite
         uses: actions/upload-artifact@v4

Makefile

@@ -74,6 +74,12 @@ data:
 	mv policyengine_us_data/storage/enhanced_cps_2024.h5 policyengine_us_data/storage/dense_enhanced_cps_2024.h5
 	cp policyengine_us_data/storage/sparse_enhanced_cps_2024.h5 policyengine_us_data/storage/enhanced_cps_2024.h5
 
+data-local-area: data
+	LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/cps/cps.py
+	LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/puf/puf.py
+	LOCAL_AREA_CALIBRATION=true python policyengine_us_data/datasets/cps/extended_cps.py
+	python policyengine_us_data/datasets/cps/local_area_calibration/create_stratified_cps.py 10500
+
 clean:
 	rm -f policyengine_us_data/storage/*.h5
 	rm -f policyengine_us_data/storage/*.db

changelog_entry.yaml

@@ -0,0 +1,8 @@
+- bump: minor
+  changes:
+    added:
+    - Sparse matrix builder for local area calibration with database-driven constraints
+    - Local area calibration data pipeline (make data-local-area)
+    - ExtendedCPS_2023 and PUF_2023 dataset classes
+    - Stratified CPS sampling to preserve high-income households
+    - Matrix verification tests for local area calibration