Open
Conversation
* Add code to build lakes layer * changes from pre-commit check * feat: domains schema * change domain and crs to use enum values * moved schema directory under hydrofabric_builds * change xarray version and correct --domain argument * add domain arguement to readme file --------- Co-authored-by: Quercus Hamlin <qhamlin@asrcfederal.com>
* feat: added support for building references to usbr reservoirs * docs: added verbiage in notebook * patch: addressed PR comments, added hydrolocations table support * patch: updating docs at top of file with new usage * patch: fixed typo
* feat: readme image * img size * center * fix corners * one more time with feeling --------- Co-authored-by: Quercus Hamlin <qhamlin@asrcfederal.com>
…e variable to the state
* initial commit: building network graph * feat: updated build graph to use polars * tests: updated existing tests * tests: added local file for testing the build graph function * patch: addressed PR comments about hydrofabric functions outside of airflow * docs: updated docstrings
…gpkg, and updated config locations (#19)
…rivers to run upstream agg/disagg functions (#20)
* initial commit: added initial upstream tracing from cookbook * feat: updated catchment classification rules to be in-line with what is needed for NGWPC HF * tests: updated tests and validity * feat: added function into hfrunner * patch: removed lingering = * nit: updated based on PR requests
* initial commit: updated README with schema * patch: updated spelling * Update README.md
#23) * feat: added aggregation of geometries * patch: tracing function not accounting for aggregation and 3+ streams * update: committing local code rather than stashing * patch: changed trace to use mainstemlp * feat: refactoring the aggregation to mapping and then reducing * feat: finished tests for id ranges and making new hfrunner additions * feat: added tests for all mapping functionalities * tests: finished integration testing * Update src/hydrofabric_builds/hydrofabric/build.py Co-authored-by: Quercus Hamlin <75846376+quercoak@users.noreply.github.com> * patch: moved schemas into _schemas_ module --------- Co-authored-by: Quercus Hamlin <75846376+quercoak@users.noreply.github.com>
* adding usgs gages, TXDOT, CADWR, ENVCA, nwm_calibraitions, etc * adding live gages from shapefiles * Ignore IDE files: .idea and *.iml * adding user local dir on top to not having a name on all paths * removing packages that were not used anymore * got rid of fiona * got rid of fiona from .toml too * adding a doc for how to run in POI gages builder * adding pytests * adding pytests * checking ruff * removing commented out lines
#38) * feat: patch for minor flowpaths in single upstream * feat: patch for minor flowpaths in single upstream * patch: reformatted tracing rules to ensure edge cases are handled * patch: updated rule 7 to not include minor segments * patch: removed unnecessary connnector swaps for aggregation * patch: added no_divide_connectors * patch: reordered rules * patch: simplified rules for connectors * patch: fixed no_divide_connectors not processing * adding usgs gages, TXDOT, CADWR, ENVCA, nwm_calibraitions, etc (#36) * adding usgs gages, TXDOT, CADWR, ENVCA, nwm_calibraitions, etc * adding live gages from shapefiles * Ignore IDE files: .idea and *.iml * adding user local dir on top to not having a name on all paths * removing packages that were not used anymore * got rid of fiona * got rid of fiona from .toml too * adding a doc for how to run in POI gages builder * adding pytests * adding pytests * checking ruff * removing commented out lines * patch: fixed bug for empty geometries * tests: updated imports * tests: fixed mocks for test_mapping * tests: fixed pytest suite * tests: added better test for cycles, and cycle detector * cicd: ran lints * tests: fixing runner integration test * cicd: update to test dir * cicd: working on tests * cicd: working on tests * cicd: working on tests * cicd: working on tests * cicd: working on tests * feat: patch for minor flowpaths in single upstream * patch: reformatted tracing rules to ensure edge cases are handled * patch: updated rule 7 to not include minor segments * patch: removed unnecessary connnector swaps for aggregation * patch: added no_divide_connectors * patch: reordered rules * patch: simplified rules for connectors * patch: fixed no_divide_connectors not processing * patch: fixed bug for empty geometries * tests: updated imports * tests: fixed mocks for test_mapping * tests: fixed pytest suite * tests: added better test for cycles, and cycle detector * cicd: ran lints * tests: fixing runner integration test * cicd: update to test dir * cicd: working on tests * cicd: working on tests * cicd: working on tests * cicd: working on tests * cicd: working on tests * cicd: working on tests --------- Co-authored-by: Farshid Rahmani <59573719+FRahmani368@users.noreply.github.com>
* snapping gages and flowpaths * organizing the code and adding pytest * ruff checked * switchingfrom requests to httpx * pytest check 1 * adding docs * adding information about EPSG:6933 * modifying the code based on review comments * argparser is activated
* initial commit: fixed graph dfs check to use rustworkx * patch: created a valid geometry filter * tests: added tests for validating geometries * tests: cleaned up comment * PR comments: addressed vectorized loop
* initial commit: changing pandas to polars * feat: updated graph builds to use polars * feat: added polars to aggregation, updated tests * nit: cleaning up comments
…tream connectivity
* initial commit. Renamed virtual flowpaths to non_nextgen_flowpaths * feat: added a sample case for the virtual_flowpaths layer * feat: added all virtual layers for the sample dataset * initial commit. Renamed virtual flowpaths to non_nextgen_flowpaths * feat: added a sample case for the virtual_flowpaths layer * feat: added all virtual layers for the sample dataset * tmp: removing existing trace and map tests as they aren't checking new virtual fp data * tests: added test data for upstream no divide fp * tests: running CONUS virtual flowpaths * patch: removed no-divide polygon cases * patch: fixed another no-divide stream order 3 * test: added coastal case for no-divide outlet * patch: fixed non-deterministic flowpath/nexus IDs and ordering of hydrofabric outputs. Build Hudson test case * tests: added sioux falls and braided rivers cases
* feat: fixed id dup issue, added percentage area contributions to the virtual flowpaths * patch: anamoly in VPU 14 * feat: fixed id dup issue, added percentage area contributions to the virtual flowpaths * patch: anamoly in VPU 14 * patch: adjusted mountain case * feat: fixed id dup issue, added percentage area contributions to the virtual flowpaths * patch: anamoly in VPU 14 * patch: adjusted mountain case * patch: resolved divide area mistakes, duplicate classifications, and hydroseq errors * patch: fixed test cases, solved bugs for connectors, solved bugs for drainage area * tests: fixed tests and ID offset
* patch: outlet IDs that have a DA < 0.0005 sqkm * patch: updated anomoly to remove false virtual fp * patch: ensured f64 are cast to int64
* nit: renaming streamorder to stream_order * patch: updated fp attrs to have new change * patch: updated pydantic schema
* bounty: updated README.md with new data model * patch: changed the schema fp attribues to include missing attributes * patch: updated reference verbiage
* patch: gages use ref fps * Bounty/readme: added new schema docs to README.md (#92) * bounty: updated README.md with new data model * patch: changed the schema fp attribues to include missing attributes * patch: updated reference verbiage * patch: gages use ref fps --------- Co-authored-by: Tadd Bindas <taddbindas@gmail.com> Co-authored-by: taddyb <tkb5476@psu.edu>
* patch: riverml variables * patch: test dir * docs: updated docs to include r_path --------- Co-authored-by: taddyb <tkb5476@psu.edu>
* patch: allow 10 percentile twi * patch: update example config to show twi options
* patch: quartz, lat, lon * patch: mfsno spelling * patch: mp and loging
…e confusion (#105)
* initial commit: added quickstart docs for pulling data * patch: added uv run command * patch: fixed cp mistakes * patch: added try except
* flowpath crosswalk * fixing some small issues * adding pytests for fp_crosswalk * checking ruff and fixing them * fixing pytest issue * adding if condition to check fp_crosswalk file exists * flowpath crosswalk * fixing some small issues * adding pytests for fp_crosswalk * checking ruff and fixing them * fixing pytest issue * adding if condition to check fp_crosswalk file exists * patch: changed flowpath code to buffer intersections to simplify procedure * simplifying the crosswalk code * adapting pytests with new changes * adding parquet file to output * reducing the min percentage to 0.005 * converting float fp_ids to int64 * making tasks to True in config * adding , if_exists=replace to fp_Crosswalk * making tasks True in config * Update configs/example_config.yaml Co-authored-by: Quercus Hamlin <75846376+quercoak@users.noreply.github.com> * Update configs/example_config.yaml Co-authored-by: Quercus Hamlin <75846376+quercoak@users.noreply.github.com> * Update src/hydrofabric_builds/schemas/hydrofabric.py Co-authored-by: Quercus Hamlin <75846376+quercoak@users.noreply.github.com> * Update src/hydrofabric_builds/schemas/hydrofabric.py Co-authored-by: Quercus Hamlin <75846376+quercoak@users.noreply.github.com> * Update configs/example_config.yaml Co-authored-by: Quercus Hamlin <75846376+quercoak@users.noreply.github.com> * cleaning up the config and pytest issues * removing double crosswalk task call --------- Co-authored-by: taddyb <tkb5476@psu.edu> Co-authored-by: Quercus Hamlin <75846376+quercoak@users.noreply.github.com>
* patch: added doc changes, schema, and naming conventions * nit: addressed pr comments
* adding CIROH_UA source file to gages * modifying config file * modifying docs
* docs: linked quickstart to README * docs: added more links to other docs
* modifying docs * updating the numbers * numbering gets corrected
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Formal delivery was documented by raytheon as being the following location: NOAA-OWP/NextGen-Water-Prediction-Capability#2
All code related to building the NGWPC Hydrofabric (NHF)
Getting Started
This repo is managed through UV and can be installed through:
Quickstart
See here for all steps for downloading data from the RTX EDFS Test Account Bucket
Development
To ensure that hydrofabric-builds follows the specified structure, be sure to install the local dev dependencies and run
uv run pre-commit installSchema