Skip to content

Time series and rprof data gathering is slow #36

@amorison

Description

@amorison

From @RobertoAgrusta

When working with a large number of profiles and time series, gathering these data takes time and memory (even when using the HDF5 output format).

Improving the parsing time and memory usage when requesting the entire data via the sdat.tseries and sdat.rprof accessors will be difficult since we already use the fast pandas and h5py parsers.

However, a possible workaround is to implement partial parsing of these data when using the steps[n].rprof and steps[n].timeinfo accessors.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions