Skip to content

Add Hdf5 serialization functions for read and write data objects#221

Merged
ethanplunkett merged 36 commits intobirdflow-science:mainfrom
chenyangkang:hdf5_serialization
Sep 29, 2025
Merged

Add Hdf5 serialization functions for read and write data objects#221
ethanplunkett merged 36 commits intobirdflow-science:mainfrom
chenyangkang:hdf5_serialization

Conversation

@chenyangkang
Copy link
Copy Markdown
Contributor

PR for issue #220

@chenyangkang chenyangkang changed the title Add Hdf5 serialization functions for read and write data object Add Hdf5 serialization functions for read and write data objects May 25, 2025
@chenyangkang
Copy link
Copy Markdown
Contributor Author

chenyangkang commented May 25, 2025

@ethanplunkett Hi Ethan, can we consider merging this after merging the other PR? I'm working on combining the multi-source data and these functions seem to be essential if we want them in hdf5 format. Please let me know if there are any concerns.

@chenyangkang
Copy link
Copy Markdown
Contributor Author

@ethanplunkett Revised based on you suggestions:

  1. add identical check for object read and write. The objects should be exactly the same after dumping and reloading.
  2. replace the read and write function for the BirdFlowRoutes and Routes with a generic function.
  3. Lint.

@ethanplunkett
Copy link
Copy Markdown
Contributor

@chenyangkang I updated the tests for serialization:

  • Use a temp directory to write file
  • Properly test (and now fail) writing NULL values

@ethanplunkett
Copy link
Copy Markdown
Contributor

I fixed the tests so it's actually testing with a NULL list, but now it's failing. I think we should just drop those tests as I'm not aware of anytime that we'll be writing NULL values.

@ethanplunkett ethanplunkett merged commit 7fe9b72 into birdflow-science:main Sep 29, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants