Read other file types for frames

[cleterrier]

HI, congrats on the 0.10 version!

I have a question about using an extra piece of code to directly read Nikon nd2 files as input for frames. Right now I have to convert all acquisitions to tif files and it is adding a significant overhead both in terms of time, hard drive space and transfer of data between computers.

There are nd2 reader in python, like nd2 reader (https://rbnvrw.github.io/nd2reader/) and pims-nd2 (https://github.com/soft-matter/pims_nd2). My question is, how difficult would it be to interface these with DECODE so that it can load frames from nd2 without the tif intermediate? I'm not suggesting to have the function built-in in DECODE (that would be nice though :) but general guidance on if/how this could be done so that I can start clumsily hacking my way through it. Thanks!

[Haydnspass]

That should be fairly easy. The only thing decode expects is a torch Tensor of size N x H x W. If you want a memory map, it could be a bit more advanced (i.e. if the whole framestack does not fit into RAM at once).

E.g. this is the relevant line for the tiff file:

DECODE/decode/utils/frames_io.py

Lines 44 to 45 in e93f1ea

	im = tifffile.imread(str(p), multifile=multifile)
	frames = torch.from_numpy(im.astype('float32'))

Without trying a drafty implementation could be

from pims import ND2_Reader

def load_nd2(file):
   frames = ND2_Reader(file)
   return torch.from_numpy(frames[:])  # maybe some datatype conversion necessary

[cleterrier]

Just to be sure, the order used by DECODE is indeed (frame, y, x), not (frame, x, y) ?

[Haydnspass]

In terms of decode's convention yes it is frame, x, y.
I can have a look at the API this week and implement it then.

[Haydnspass]

Already done yourself? Otherwise, could you upload me an example file somewhere?

[cleterrier]

I got it working in a batch fit notebook. I can even directly fit while reading nd2 files off the server, so no transfer and conversion needed anymore!

[Haydnspass]

Could you still upload a small (dummy) file? Maybe it is of use for other people as well :)

[cleterrier]

Hi @Haydnspass, I published the batch fitting notebook able to process nd2 files and an example nd2 sequence (256x256 pixels, 100 frames) here: https://github.com/cleterrier/DECODE_NC

[cleterrier]

The code uses nd2reader (https://github.com/rbnvrw/nd2reader), the latest 3.3.0 release that correctly assigns t vs z dimensions, so the code works as intended withhout the "z instead of t" hack I used previouly.

[christian-7]

Hi @cleterrier,
I am trying to use your nice batch script, but the nd2 reader seems to have a different memory management. I keep getting this error (pasted below), no matter how small I make the batch size. I have 8Gb VRAM, but it seems I can not even allocate 384 Mb. Have you seen this before? The tiff reader from decode (frames = decode.utils.frames_io.load_tif(frame_path)) works fine, but I would like to use nd2 directly.
Thanks

RuntimeError: CUDA out of memory. Tried to allocate 384.00 MiB (GPU 0; 8.00 GiB total capacity; 6.15 GiB already allocated; 0 bytes free; 6.52 GiB reserved in total by PyTorch)

[cleterrier]

I'm not sure, I remember getting this error if I didn't clear the memory from a previous step (running a training or an inference)

[christian-7]

Yes, the memory is always full. I need to dig into how to use/clear this more efficiently. For now, it seems to work with pims-nd2. I only changed one line in your script.