chore -cached loading (V3 omezarr support) step one by froyo-np · Pull Request #205 · AllenInstitute/vis

froyo-np · 2025-10-13T21:31:07Z

Cached Fetching via zarrita store

What

caches fetches for byte-ranges, which our test-browsers seem unwilling to do
runs those fetches on a pool of workers
hides all the above from existing rendering code via a 'decoder' passed to the renderer
[ ]

How

Cherry-picked worker-pool, worker implementation, and priority-based data store from @Jarbuckle 's excellent work
Added the cache-injected decoder - see cache-lower.ts

Known issues

the blosc decompression still happens on the main thread, due to unforseen complications with how zarrita handles sharded data.
fetches for byte-ranges are never cancelled, due to what seems to be a mistake in how zarrita handles sharded fetches...
we still dont know how to ship a web-worker whole-sale from an imported package we build with vite, so for now, the worker has to be created in a somewhat awkward way by the user of this work, (in our case, the demo).

Screenshots

This section is optional if there are no visible changes

If possible add screenshots of the visible additions in the UI.
If there are changes in the UI, add Before and After Screenshots for quick overview.
If there was a Figma design, add a link to that here as well.
Hint : Drag and Drop any images you want to add to the PR. Also you can create a gif of an interactive version and add that!

PR Checklist

froyo-np · 2025-10-13T21:32:32Z

packages/omezarr/src/zarr/cache-lower.ts

+import { logger } from '@alleninstitute/vis-core';
+import { ZarrFetchStore, type CachingMultithreadedFetchStoreOptions } from './cached-loading/store';
+
+export function decoderFactory(url: string, workerModule: URL, options?: CachingMultithreadedFetchStoreOptions) {


this file is the only (interesting) non-cherry-picked change here - the rest is directly plucked from #203

froyo-np · 2025-10-13T21:42:48Z

site/src/examples/omezarr/omezarr-client.tsx

    };
    useEffect(() => {
        if (cnvs.current && server && !renderer) {
+            const { decoder } = decoderFactory(getResourceUrl(props.res), WORKERS);


here is where we create the decoder - we create it when we are about to load the data... now that i'm looking at this, it wont ever re-create the decoder (like say, when you want to view a different file), but thats been an issue forever in this demo, so maybe its fine

Jarbuckle

Just a couple small things!

packages/omezarr/src/zarr/cache-lower.ts

Jarbuckle · 2025-10-13T21:43:10Z

site/src/examples/omezarr/omezarr-client.tsx

    };
    useEffect(() => {
        if (cnvs.current && server && !renderer) {
+            const { decoder } = decoderFactory(getResourceUrl(props.res), WORKERS);


Are we concerned at all about the prospect of useEffect() calls happening multiple times and producing a lot of workers? Will the old workers be discarded by the GC?

excellent question - yes we should be concerned... i dont think that is happening in this very canned demo... but we should consider preventing that scenario!

as for workers being GC'd - I dont think that happens... workers have to be killed externally with terminate() I think. I'll see about how we would do that in a sane way....

Jarbuckle · 2025-10-13T21:59:36Z

Set it up locally and tested, working just great!

…rkerPool

packages/omezarr/src/zarr/cached-loading/fetch-data.worker-loader.ts

packages/omezarr/src/zarr/cached-loading/store.ts

Jarbuckle

Looks good to me! Works well, seems to cover what we need it to cover! Woo!

Jarbuckle

Hold that -- something's not quite working right: I'm trying to view the example OME-Zarr in the Priority Cache example page and it's not loading 😟

Jarbuckle

Works!

froyo-np added 3 commits October 13, 2025 14:05

make it at least sort of work

2157f86

tidy up

256767c

demo

87cc1f9