Parsing pyiron HDF5 files

[jan-janssen]

I updated the examples for LAMMPS, SPHInX and VASP to directly read the HDF5 files to a dictionary and afterwards write the dictionary to an HDF5 file again. The LAMMPS and SPHInX examples both work on mybinder:

• https://github.com/pyiron-dev/pyiron-hdf5-format/blob/main/lammps.ipynb
• https://github.com/pyiron-dev/pyiron-hdf5-format/blob/main/sphinxdft.ipynb
• https://github.com/pyiron-dev/pyiron-hdf5-format/blob/main/vasp.ipynb

The process is the same for all three:

1. A calculation is set up and executed.
2. The content of the whole job including the output is read to a dictionary.
3. The output schema is extracted, this schema is required to create the HDF5 file later on and should be developed by the person creating the job_type.
4. The original HDF5 file is deleted and a new HDF5 file is written based on the schema for the particular job.
5. The job object is reloaded from the newly written HDF5 file and the input and output are validated.

Does this address the needs you had in terms of easy access to the pyiron HDF5 files?

EDIT: Add VASP example.

[Tara-Lakshmipathy]

Yes, I think having an easily accessible schema to create the HDF5 file would be useful indeed. The dictionary which gets created earlier would also be helpful for mapping to the schemas of the data systems easily. @niklassiemer or @pkruzikova may have something more to add to this, but I'm quite satisfied.
There was another idea we were playing around with: adding the conda environment the job was run on to the HDF5 files. I don't see this as being particularly helpful for the original HDF5 file that is generated. But if we could have the environment information in the new HDF5 file, that would be awesome. Something like conda env export > environment.yml, but instead of yml, it's written into the HDF5 file.

[jan-janssen]

The corresponding fixes are going merged intopyiron_base and then we can separate the io part into a separate package:
pyiron/pyiron_base#1469

[jan-janssen]

I released a new version of h5io_browser, while the update for pyiron_base is still pending pyiron/pyiron_base#1470 these recent changes drastically reduce the code required for the examples:

• https://github.com/pyiron-dev/pyiron-hdf5-format/blob/main/lammps_h5io.ipynb
• https://github.com/pyiron-dev/pyiron-hdf5-format/blob/main/sphinxdft_h5io.ipynb

[jan-janssen]

The classes are now moved to https://github.com/pyiron/pyiron_dataclasses