Cannot write h5 file

[andrewlhicks]

Hello,

I got the same error when attempting to write an h5 file as did another user (issue #4388):

  File "/data/andrewhi/q-tensor-3d/q3d/saves.py", line 63, in save_checkpoint
    with CheckpointFile(path,'w') as file:
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/andrewhi/venv-firedrake/lib/python3.12/site-packages/firedrake/checkpointing.py", line 5
38, in __init__                                    
    self.set_attr_byte_string("/", "dmplex_storage_version", version)
  File "/data/andrewhi/venv-firedrake/lib/python3.12/site-packages/firedrake/checkpointing.py", line 1
700, in set_attr_byte_string
    grp = self.require_group(path)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/andrewhi/venv-firedrake/lib/python3.12/site-packages/firedrake/checkpointing.py", line 1
780, in require_group                              
    _self = self.h5pyfile 
            ^^^^^^^^^^^^^ 
  File "/data/andrewhi/venv-firedrake/lib/python3.12/site-packages/firedrake/checkpointing.py", line 1
653, in h5pyfile        
    self._h5pyfile = h5i.get_h5py_file(self.viewer) 
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
  File "firedrake/cython/hdf5interface.pyx", line 35, in firedrake.cython.hdf5interface.get_h5py_file
  File "h5py/h5i.pyx", line 34, in h5py.h5i.wrap_identifier
  File "h5py/h5i.pyx", line 39, in h5py.h5i.wrap_identifier
  File "h5py/defs.pyx", line 1222, in h5py.defs.H5Iget_type
RuntimeError: Unspecified error in H5Iget_type (return value <0)

I noticed the other discussion thread was never resolved. I feel that the solution here is either to install a newer version of HDF5 or an older version of h5py. The only issue is I don't really know how to do this in practice, even after reading over this page. My OS is Ubuntu 24.04.2 LTS.

[connorjward]

To downgrade h5py you should run pip install -U h5py==VERSION where VERSION is some older version.

How recently did you install?

Another possible fix might be to uninstall h5py, run pip cache remove h5py, then reinstall.

[connorjward]

I forgot that you also need to set HDF5_MPI=ON.

That error is quite strange. I have no experience with ROCm I'm afraid.

If this still doesn't work I would recommend:

• apt uninstall libhdf5-mpi-dev
• Reinstall PETSc but pass --download-hdf5 to configure
• export HDF5_DIR=$PETSC_DIR/$PETSC_ARCH
• pip cache purge
• pip install Firedrake again

[andrewlhicks]

The first option didn't work. I don't have root access, so neither does the second one.

Maybe I could do a local install of HDF5 and link to that one instead? How do you do that?

[connorjward]

To have a local install just follow the instructions above (set --download-hdf5 etc). Just omit the apt instruction. I only included that because you're less likely to hit issues if you only have one HDF5 on your system. It should still work.

[andrewlhicks]

So I tried the last three lines of your solution (since I had already passed the --no-package-manager to configure). It seems to have worked. So the solution would then be, assuming one has passed --no-package-manager or --download-hdf5 to configure and installed petsc:

$ export HDF5_MPI=ON
$ export HDF5_DIR=$PETSC_DIR/$PETSC_ARCH
$ pip cache purge
$ pip install --no-binary h5py 'firedrake[check]'

[connorjward]

Ah I didn't realise you had passed --no-package-manager. Setting the right HDF5_DIR was probably the solution all along.

Glad it's sorted.