Caching

[rabernat]

We likely want to cache the upstream chunks.

The easiest thing to do is use a Zarr LRU cache to wrap the store: https://zarr.readthedocs.io/en/stable/api/storage.html#zarr.storage.LRUStoreCache

This should persist as long as the fastapi process stays alive.

The next step would be to recreate this as a distributed cache using Redis for a solution that scales to multiple processes / instances (e.g. lambda).