[BUG] Folder of 2D TIFs downsamples much slower than 3D TIF from Napari brainreg

[marcodallavecchia]

Describe the bug
For the same data (no matter the size), registering with brainreg through Napari, a folder of 2D TIFs takes much longer (since they are loaded lazily) than a 3D TIF (loaded entirely in memory).

I believe this is due to this line, where depending on how Napari passes the data through its layers, we have a clear slowdown.

To Reproduce
Take same image as 3D stack or saved as folder of 2D TIFs (I used ImageJ Image Sequence export)

Expected behaviour
If there's enough memory both scenarios could force all images to be loaded in memory. If not the usage of dask, as described here should solve the issue.

[adamltyson]

Thanks for raising this @marcodallavecchia. Do you have any quantification about how much slower it is?

[marcodallavecchia]

@adamltyson I didn't test it systematically, but it's quite significant in my opinion.

I used down-sampled whole brain to 25um (~150Mb) registered to the allen mouse 50um on my laptop saved locally on my disk.

• 3D TIF loaded entirely in memory: downsampling xy time: ~1s

• Plane-by-plane lazy loaded: downsampling xy time: 1min 9s

[alessandrofelder]

Testing reading speed on my desktop on a large dataset that fits into memory (17GB, 5 um isotropic MS cx left background channel) registering to allen_mouse_25um

The 3d tiff version took:

Downsampling data in x, y
2550it [01:42, 25.00it/s]
Downsampling data in z
338it [00:11, 28.94it/s]
filtering: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 456/456 [00:04<00:00, 101.55plane/s]
filtering: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 432/432 [00:03<00:00, 135.70plane/s]

I will post the results here once they finish, but the folder of 2D tiffs is currently doing <1.5 iterations per second on downsampling in x/y (~20x slower), which isn't great. But it's not technically a full bug and given that we tend to try to minimise work on brainreg in favour of brainglobe-registration, this is maybe just something useful to know about reading arrays from napari Layers - cc @IgorTatarnikov in case this is relevant for brainglobe-registration (but I know you've already put a lot of effort into performance there!)

[IgorTatarnikov]

I've noticed some slow downs in brainglobe-registration which may be related. Rotating the atlas and rendering it used to be "snappier". Something to be investigated!

[alessandrofelder]

For completeness, 2D tiff folder results:

Downsampling data in x, y
2550it [31:10,  1.36it/s]
Downsampling data in z
338it [00:11, 29.84it/s]
filtering: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 456/456 [00:04<00:00, 113.18plane/s]
filtering: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 432/432 [00:03<00:00, 135.27plane/s]
16:18:00 : INFO : Dummy-1 : Registering

Seems like it's specifically the in-plane downsampling that is a lot slower (which is expected).

For context, the entire brainreg run took

• 11 mins for the 3D tiff case
• 33 mins for the folder of 2D tiffs case

so we are incurring a significant performance penalty

[adamltyson]

Thanks @alessandrofelder. Looks like we should probably take a look at this sometime. I assume we can improve this without loading everything into memory, I assume there should be a quicker way to load & downsample the data.

Was this with the napari plugin or the CLI?

[alessandrofelder]

With the napari plugin. Our suspicion is that it's the combination of napari trying to be clever with lazy loading the 2D tiffs and our reading from the napari layer that is slowing things down.

[alessandrofelder]

With the brainreg CLI, the folder of 2D tiffs is twice as fast (but both are pretty quick: 3 and 1.5 minutes)!

It looks like we parallelise the loading of 2D tiffs in the main Python function called by the CLI but not in the napari plugin.

3D tiff:

$ time brainreg ~/data/test-brainreg-reading-speed/stack_3d.tif ~/test-brainreg-reading-speed-3d-cli/ --orientation prs -v 5 5 5 --atlas allen_mouse_25um
...
DEBUG:root:Downsampling stack in X/Y
100%|███████████████████████████████████████| 2550/2550 [01:41<00:00, 25.02it/s]
DEBUG:root:Converting downsampled stack to array
DEBUG:root:Downsampling stack in Z
filtering: 100%|██████████████████████████| 456/456 [00:04<00:00, 113.83plane/s]
filtering: 100%|██████████████████████████| 432/432 [00:03<00:00, 135.81plane/s]
...
INFO:root:Finished. Total time taken: 0:03:13.603421
                       INFO     2025-08-22 11:14:18 AM - INFO -       cli.py:305
                                MainProcess cli.py:305 - Finished.              
                                Total time taken: 0:03:13.603421                

real	3m14.333s
user	8m54.905s
sys	0m28.157s

Folder of 2D tiffs:

$ time brainreg ~/data/test-brainreg-reading-speed/stack2d/ ~/test-brainreg-reading-speed-2d-folder-cli/ --orientation prs -v 5 5 5 --atlas allen_mouse_25um
...                             
DEBUG:root:Determining the maximum number of CPU cores to use
DEBUG:root:Number of CPU cores available is: 18
DEBUG:root:Setting number of processes to: 18
INFO:root:Loading raw image data
                       INFO     2025-08-22 11:15:56 AM - INFO -       main.py:48
                                MainProcess main.py:48 - Loading raw            
                                image data                                      
DEBUG:root:Data type is: directory of files
DEBUG:root:Determining the maximum number of CPU cores to use
DEBUG:root:Number of CPU cores available is: 18
DEBUG:root:Setting number of processes to: 18
Loading images: 100%|██████████████████████| 142/142 [00:19<00:00,  7.32plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:19<00:00,  7.32plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:19<00:00,  7.20plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:19<00:00,  7.17plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:19<00:00,  7.11plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  7.09plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  7.07plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  7.04plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  7.01plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  6.99plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  6.98plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  6.97plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  6.87plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  6.86plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  6.85plane/s]
Loading images: 100%|██████████████████████| 136/136 [00:20<00:00,  6.53plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  6.81plane/s]
Loading images: 100%|██████████████████████| 142/142 [00:20<00:00,  6.80plane/s]
filtering: 100%|██████████████████████████| 456/456 [00:04<00:00, 104.62plane/s]
filtering: 100%|██████████████████████████| 432/432 [00:03<00:00, 125.37plane/s]
...
INFO:root:Finished. Total time taken: 0:01:34.126559
                       INFO     2025-08-22 11:17:28 AM - INFO -       cli.py:305
                                MainProcess cli.py:305 - Finished.              
                                Total time taken: 0:01:34.126559                

real	1m34.897s
user	12m32.248s
sys	0m59.200s