Skip to content

HDF5 Plugin Working Group

Jump to bottom
Dana Robinson edited this page Jul 30, 2024 · 17 revisions

The HDF5 Plugin Working Group meets monthly, normally on the second Thursday of the month at 10 am Central US time (the normal time for the HDF5 Working Group meeting).

This meeting is for the sustainability and governance of all HDF5 plugins - filter/compression, virtual file driver (VFD), and virtual object layer (VOL) connector. Anyone who is interested in this topic or wants to contribute their time and energy is welcome to attend. This meeting is NOT intended for providing technical support. Please use the forum for that.

Zoom link: https://us06web.zoom.us/j/89601195963

The agenda and any cancellations will be posted below and on the forum, usually on Monday. If you have any action items to discuss, please email derobins at hdfgroup dot org to get them added to the agenda. If there are no pressing issues by Monday, the meeting may be cancelled.

15 August 2024

I am out of the office on the second Thursday of the month, so the meeting will be moved to the 15th.

18 July 2024

Agenda

  • Governance (where put docs? - wiki?)
  • Who wants to contribute?
    • Allen, me
  • Repo reorganization
  • Supported platforms and compilers
  • CI - Do we have enough?
  • How do we manage compression libraries? (more challenging than managing filter code)
  • Do we need to add any new plugins for the 1.14.5 release?

Overall Goals

  • One-stop shopping for official filters
  • Maintain high quality filters
    • Alignment with h5py, netCDF, BLOSC, Conda, Debian, other installers
      • How do we deal with Matlab, IDL, older versions, etc.?
    • Uniform install locations, or at least best practices
    • How handle versioning?
  • Clearly indicate which filters are "highly supported"
  • Long-term HDF5 file maintenance (storing filter provenance)
  • Improve documentation

Random notes

  • Many versions of filters in the wild, which is a problem (Elena)
  • Need to support more filters (Elena)
  • Two parts to a filter - filter itself and compression library (Allen)
    • This repo is only about filters, not libraries
  • Need guidance for what constitutes a "proper filter"
    • Code quality, examples, etc.
    • Could go over this as a part of assigning a filter ID number
    • Though this shouldn't be mandatory (some filters may be private)
  • Need to move the filter ID list here
    • Turn it into a matrix w/ person responsible, supported compression library version(s), etc.
  • Cross-platform builds are difficult for some filters
    • Is this okay?
    • Need to document supported compilers and platforms
    • Do we support the Autotools?
      • I hope not (Dana)
  • Two levels of filters
    • Special filters that we carefully maintain and care about (h5py filters, etc.)
    • Community filters that are best-effort
  • What about external software like hsds? How do we help them deal with compressed HDF5 data?
  • Which filters are maintained/community?
    • Community: MAFISC, SZ
    • Maintained: Everything else

To-Do

  • Create basic documentation about purpose of repo, etc.
  • Separate existing filters into "highly maintained" and "community"
  • Make a list of existing filters that should be added to the "community" section
  • Maintain filter ID list via this repo
Clone this wiki locally