Review IO pattern for cellvoyager converter

[tcompa]

While profiling Fractal tasks for different purposes, I observed the following information for a converter task running on SLURM

$ sacct -j 4561 --format 'JobID,JobName%40,Start,End,Elapsed,CPUTime,TotalCPU,NCPUS,MaxDiskRead,AveDiskRead,MaxDiskWrite,AveDiskWrite'

JobID                                         JobName               Start                 End    Elapsed    CPUTime   TotalCPU      NCPUS  MaxDiskRead    AveDiskRead MaxDiskWrite   AveDiskWrite 
------------ ---------------------------------------- ------------------- ------------------- ---------- ---------- ---------- ---------- ------------ -------------- ------------ -------------- 
4561                  Convert_Cellvoyager_to_OME-Zarr 2025-03-28T14:05:05 2025-03-28T14:05:15   00:00:10   00:00:10  00:32.926          1                                                         
4561.batch                                      batch 2025-03-28T14:05:05 2025-03-28T14:05:15   00:00:10   00:00:10  00:00.036          1        0.04M          0.04M        0.00M          0.00M 
4561.0                                         python 2025-03-28T14:05:05 2025-03-28T14:05:15   00:00:10   00:00:10  00:32.889          1     1405.37M       1405.37M      378.99M        378.99M 

I am curious about the meaning of
AveDiskRead (Average number of bytes read by all tasks in job): 1405.37M
AveDiskWrite (Average number of bytes written by all tasks in job): 378.99M

Context:

• SLURM job 4561 is the compute part of the converter compound task
• This is a conversion of a single well in 2D.
• The multiscale image arrays take about 380M on-disk, which is compatible with AveDiskWrite
• 64 tif files are present for this given well in the input data, and they are 7.7M each. A single read of each file would result in 7.7M x 64 = 493M, rather than 1405.37M

How could this happen?

1. My first explanation was that maybe we are re-reading all images for each channel (note that there are 4 channels here) - see https://github.com/fractal-analytics-platform/fractal-tasks-core/blob/main/fractal_tasks_core/tasks/cellvoyager_to_ome_zarr_compute.py#L194-L196. But this is probably not the case, thanks to https://github.com/fractal-analytics-platform/fractal-tasks-core/blob/main/fractal_tasks_core/tasks/cellvoyager_to_ome_zarr_compute.py#L165-L168.
2. Another explanation could be that we first write to disk and then build pyramids. To be verified.
3. Another explanation could be that we are misusing dask somewhere, and re-computing multiple times. To be verified.

I don't have a conclusive take here, but if this is one of the task that will be soon ported to ngio it'd be worth double-checking that the file-access patterns are not redundant.