Fix: upload datasets to public HuggingFace repo (#280)

* fix: upload datasets to public HuggingFace repo

The push workflow uploads to policyengine/policyengine-uk-data-private
but policyengine-uk downloads from policyengine/policyengine-uk-data.
This means new columns (like highest_education) never reach downstream
consumers. Fix by also uploading to the public repo.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: pass version parameter to public repo upload

upload_files_to_hf requires a version parameter for tagging.
Without it the upload would fail with TypeError at runtime.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Author: MaxGhenis

changelog_entry.yaml

@@ -0,0 +1,4 @@
+- bump: patch
+  changes:
+    fixed:
+    - Upload datasets to public HuggingFace repo (policyengine/policyengine-uk-data) in addition to private repo, so policyengine-uk gets the latest data.

policyengine_uk_data/storage/upload_completed_datasets.py

@@ -1,5 +1,10 @@
+from importlib import metadata
+
 from policyengine_uk_data.storage import STORAGE_FOLDER
-from policyengine_uk_data.utils.data_upload import upload_data_files
+from policyengine_uk_data.utils.data_upload import (
+    upload_data_files,
+    upload_files_to_hf,
+)
 
 
 def upload_datasets():
@@ -14,13 +19,22 @@ def upload_datasets():
         if not file_path.exists():
             raise ValueError(f"File {file_path} does not exist.")
 
+    version = metadata.version("policyengine-uk-data")
+
     upload_data_files(
         files=dataset_files,
         hf_repo_name="policyengine/policyengine-uk-data-private",
         hf_repo_type="model",
         gcs_bucket_name="policyengine-uk-data-private",
     )
 
+    # Also upload to the public repo consumed by policyengine-uk
+    upload_files_to_hf(
+        files=dataset_files,
+        version=version,
+        hf_repo_name="policyengine/policyengine-uk-data",
+    )
+
 
 if __name__ == "__main__":
     upload_datasets()