Skip to content

Comments

Invalidate shard index cache when chunk source cache is invalidated#882

Open
r2dliu wants to merge 1 commit intogoogle:masterfrom
r2dliu:fix/invalidate-shard-index-cache
Open

Invalidate shard index cache when chunk source cache is invalidated#882
r2dliu wants to merge 1 commit intogoogle:masterfrom
r2dliu:fix/invalidate-shard-index-cache

Conversation

@r2dliu
Copy link

@r2dliu r2dliu commented Feb 11, 2026

Problem:
When invalidateCache() is called on a ZarrVolumeChunkSource (e.g. after an external write to a sharded zarr3 array), only the chunk data cache is invalidated. The shard index cache remains stale.

A normal refresh fixes the issue since all caches are cleared on reload, but programmatically clearing the cache using the neuroglancer internals caused wonky behavior and garbage data loading.

This is probably a pretty niche issue and is certainly and uncommon or potentially even bad use case. This is a hacky fix that I threw together but curious what pattern the maintainers might prefer, or whether they would consider it worth addressing to begin with.

@google-cla
Copy link

google-cla bot commented Feb 11, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@chrisj
Copy link
Contributor

chrisj commented Feb 12, 2026

@r2dliu Can you explain how to reproduce this? It looks like it relies on ChunkSource.invalidateCache to be called which only seems to be called when using a PythonDataSource

@r2dliu
Copy link
Author

r2dliu commented Feb 12, 2026

Step 1: Load a zarr3 dataset from your fileserver into neuroglancer
Step 2: While still viewing this dataset in browser, manually edit said zarr3 dataset on your fileserver for a chunk you know is loaded
Step 3: My use case manually calls source.invalidateCache() for the currently selected zarr3 layer for the given chunk. For zarr2, neuroglancer will redownload the chunks and update the browser render properly. In zarr3, the chunk glitches and doesn't load the proper values, displaying garbage
Step 4: If you refresh browser the new changes will load properly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants