Skip to content
This repository was archived by the owner on Oct 2, 2023. It is now read-only.

adding manual load cpdb yml#406

Open
DeaBardhoshi wants to merge 4 commits intomainfrom
db-checkbook-load-cpdb
Open

adding manual load cpdb yml#406
DeaBardhoshi wants to merge 4 commits intomainfrom
db-checkbook-load-cpdb

Conversation

@DeaBardhoshi
Copy link

Added a manual loading YML for the CDPB historical projects + made a python script for merging different CPDB files together. Added NYCOC Checkbook YML that reads in checkbook data

@athursland
Copy link
Contributor

  • submitted nycoc_checkbook.yml in my PR so we don't need the duplicate - also, I think you need to specify the source path on your local machine (it's referencing my folder directories right now)
    path: /Users/alexandrathursland/Documents/NYC-DCP/historical-spend-data/checkbook_citywide_agencies.csv

@athursland
Copy link
Contributor

athursland commented Jul 18, 2023

regarding library/script/dcp_cpdb.py:

  • you're reading the cpdb geometries data in as pandas dataframes, combining and then writing to a CSV, but the cpdb geometries are shape files not csvs - and we originally used GeoPandas rather than pandas to concatenate them. I'm not 100% sure but this may cause an issue with interacting with a GeoPandas df down the line
  • cpdb_projects.csv doesn't need to be joined onto the cpdb geometries (cpdb geometries contains all the same info plus geometries)


info:
description: |
Checkbook NYC Spending API: https://www.checkbooknyc.com/spending-api
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would include a more detailed description, maybe also without the Checkbook NYC link (since this YAML is just for the CPDB geometries). We could include a brief overview of what CPDB is and what kind of data this is (points and polygons)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file can be deleted since nycoc_checkbook.yml was already merged to main

Copy link
Contributor

@athursland athursland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete nycoc_checkbook,yml, delete library/script/dcp_cpdb.py, update description in dcp_cpdb.yml, update path/subpath specs in dcp_cpdb.yaml to dump all of the geometries to digital ocean

@DeaBardhoshi DeaBardhoshi removed the request for review from alexrichey July 27, 2023 14:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants