switches to using parquet files that have been extracted for the model

[tomjemmett]

No description provided.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (e85a8dc) to head (205ec0b).
Report is 10 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #297   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           18        18           
  Lines          999       999           
=========================================
  Hits           999       999           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[StatsRhian]

Wasn't expecting you to respond on your day off @tomjemmett!
I'm just running QA checks against my previous runs but it should all be good.

[StatsRhian]

We're not currently able to run this on dev data. We can run it on v3.1.
It's erroring with a reshape error at the IP activity avoided calculations. YiWen & I are investigating.

Other minor changes which are suggested

• rename Model_version to Data_version
• Or grab data version from the params file directly? (v3.1 - > v3.1.0)

Once it's fixed, I intend to compare with a run I did on Thursday afternoon to ensure similar numbers before approving

[yiwen-h]

Have checked why it's not running on dev data - there are duplicates in the activity_avoidance. Example below. RAP should have been mapped to RAL (successor organisation) - so the RAP rows of activity should have been removed?

Can get round this by doing drop_duplicates but this flags an issue in nhp_data perhaps

[yiwen-h]

Ok so in summary: the notebook runs on v3.1 data. I don't think we can approve the PR until it runs on dev / v3.2 data as well. So we're awaiting the data fix. have raised an issue here: The-Strategy-Unit/nhp_data#43

[StatsRhian]

Sorry forgot one data_version