Fitting - batch_size to avoid maxing out GPU RAM - depending on frame number and size

[cleterrier]

Hi,

Usually I am fitting an acquisition of 60,000 frames of 256x256 pixels (7.7 GB as 16-bit stack) and the Fit notebook works fine with a batch_size of 32 on my GTX 1080 with 8 GB RAM (I had to drop it from 64 in the original notebook to fit within 8 GB).

Now I'm trying to fit 27,020 frames of 480x480 pixels (12.2 GB as 16-bit stack). A batch_size of 32 maxes out the RAM, however going down, even to 16 still maxes out the RAM. A batch size of 8 did the trick (which is logical as 480x480 is roughly 4x larger than 256x256)

Does RAM usage depends differently on the frame size (width X height in pixels) and on the total number of frames (here 27,020)? Can it be useful to break the sequence in smaller stacks (like 2 stacks of 13,510 frames) ? By the way I tried batch_size='auto' as argument as I read in the docs but it resulted in an error.

[Haydnspass]

Hey :)

so the RAM needed does not depend on the total length of the sequence as the sequence is broken into batches of size batch-size in the first place. This is done automatically and you don't need to split anything.
While a larger batch-size is faster, I don't think it has a significant effect in Inference mode, so I would not worry. If you use the GPU not only for DECODE but you have an actual screen attached, you might end up with varying numbers anyway. The only thing that can be tricky is if you really max out your GPU RAM utilisation with DECODE and then open a video game for example, cause that could then break the fitting.

batch_size='auto' was a bit fast, since you have seen this in the current master, but I guess you are using DECODE in the packaged version v0.9.4. I try to change the docs pointing to the latest released version instead of master by default (you can change this in the bottom left).
We will make a new release soonish, then batch_size='auto' will be available

[cleterrier]

Sounds great, thanks. Does updating as specified in the installation instruction bring DECODE to master or to the latest release? How to install the master version if not?

[Haydnspass]

No, updating as described in the docs updates to the latest release.

We change the code quite a bit every now and then, so if you want to use master you would really need to get involved in the codebase. Probably the best option with respect to this issue is to go for the batch-size you figured out and wait for the next release (~2weeks from now I think).

Otherwise, you would need to follow the DEVELOPER.md

[Haydnspass]

Btw.: Updated the docs so that it goes for the latest release (and not master) by default.