Long load times for some thumbnails

[JensWendt]

Hello,

I recently checked the website again and found that some thumbnails for datasets took longer than others to load.
I then stumbled upon my own dataset "WSI_resaved.zarr" which did not load at all, which irritated me.
After looking into it, I assume that for thumbnail generation the smallest image pyramid is opened/processed.
For datasets with very large smallest image pyramids (8167x3815 for mine and 3316x2295 for "396ac197-aeed-4121-b590-d746b5b27862.zarr" as another example) this leads to a very long thumbnail creation time.
Now the actual question:
With ome-zarr-py can I load my smallest image pyramid and create - let's say 4 - additional pyramids from it, breaking it down to a reasonable ca. 510x238?
As a next step, can I then just modify the original .zarr by adding the additional image pyramids and modifying the WSI_resaved.zarr/0/zarr.json with regards to additional datasets paths with appropriate coordinateTransformation?

Best

Jens

[will-moore]

Hi @JensWendt

Yes, even your lowest resolution at https://ome.github.io/ome-ngff-validator/?source=https://radosgw.public.os2.wwu.de/ngff/WSI_resaved.zarr/0/ has 1440 chunks, which is going to take a long time to load!

If you look at https://github.com/ome/ome-zarr-py/blob/master/docs/source/python.rst and search for "downsample_pyramid_on_disk" you'll find some code that does mostly what you want.
However, dask.to_zarr() is not yet working with Zarr v3.

One option might be to generate the additional pyramid levels on the Zarr v2 data (if you have it available somewhere) and then run the conversion again.

Another alternative is to go even further back to the original conversion to OME-Zarr v0.4. Did you use bioformats2raw? Do you remember if you chose a minimum resolution for the pyramid? "By default, the resolutions will be set so that the smallest resolution is no greater than 256x256".

Whichever method you use, you should be able to modify the zarr at that location, but if you're overwriting the whole image, just check that the only diffs are ones that you intend to make.