Combined Persistent & Cached Dataset

[elitap]

In the case of high-resolution 3D images, resampling is time-consuming, making filling the cache slow. Additionally, the entire dataset is unlikely to fit into RAM, necessitating repeated resampling for each experiment and cache replenishment if only a fraction of the data is cached.

Alternatively, the persistent dataset enables resampling to a specific resolution only once, allowing its use across multiple experiment runs. However, loading from the hard drive can be slow.

Is there a combined version of a dataset that allows preprocessing the data and storing it on the hard drive, followed by utilizing a RAM cache on this preprocessed data? While this could be achieved by manual preprocessing and using a Cacheddataset, it would require maintaining track of the original data resolution. A PersistentCached Dataset would prove quite useful for such scenarios. I am wondering if someone already worked on such a solution? As fare as I am aware, there isn't an existing solution in the dev branch?