Data import/export functionality

[smartalecH]

It would be great to have robust and standardized tooling for data import/export.

For example, it would be useful to be able to cache to disk LayerSolveResults scattering matrices, and amplitudes corresponding to a particular basis.

Pickle files aren't very robust (or secure). Perhaps Hdf5?

[mfschubert]

I have been using json for this:

https://github.com/invrs-io/totypes/blob/main/src/totypes/json_utils.py

[smartalecH]

Oh interesting... how big do these get? json isn't compressed or serialized in anyway.

[mfschubert]

They are about 1.4x the size of the object in memory. (I actually test for it: https://github.com/invrs-io/totypes/blob/main/tests/test_json_utils.py#L93)

Not a huge overhead, and the benefit is the file can be almost visually inspected.