Object Storage

Object storage

What? Implement a method to easily save/load PyEMMA high-level objects to/from disk.

Why? To facilitate interactive or scripted analysis in with large datasets. Without adaptations, the behavior of the pickle module is not well-defined because it's a priori not defined which attributes are data belonging to an object, and which are just links to other resources.

Estimators

• Estimation parameters get_params() and set_params(): Input parameters used to construct the estimator object.
• Estimation state get_state() and set_state(): State variables set by estimation. This includes estimates that connect data and model, such as convergence information.

Models

Can be mixed in to estimator or standalone.

• Model parameters get_model_params() and set_model_params(): Estimated or set parameters of the model.

Transformers, KineticModels etc.

All of these are subclass of Models and inherit the model I/O properties.

Implementation with pickle

We can implement object storage with the pickle or cpickle modules.

• __getstate__() and __setstate__() need to be overloaded in order to save/load the desired content of Estimators, Models etc.
• Does pickle have efficient protocols (compressed and fast)? Compare to np.savez_compressed

Implementation with numpy

We can implement object storage with np.savez_compressed and np.load