Clarification on Sliding Window Inference

[sh3rlock14]

Hello everybody!
I am not grasping entirely how sliding_window_inference() works, in particular:

1. I guess that by setting overlap=0 and roi_size=input_size, it would be as using SimpleInferer, right?
2. in a validation step (as in pytorch lightning) I would call directly the model on my input e.g. output = model(input), it is the same as calling output = inferer(inputs, model), with inferer=SimpleInferer, right?
3. what does sw_batch_size param refers to?
   I followed the tutorial about the post_transform but I could not figure it out what was happening when changing its value from 1 to higher values.

For the experiment I chose overlap=0 so that I could see clearly what was happening at each window, and then choose the roi_size to be equal to input size.

class ToyModel:
    # A simple model generates the output by adding an integer `pred` to input.
    # each call of this instance increases the integer by 1.
    pred = 0
    def __call__(self, input):
        self.pred = self.pred + 1
        return input + self.pred

input_tensor = torch.zeros(3, 1, 200, 200) # BCWH
output_tensor = sliding_window_inference(
    inputs=input_tensor, 
    predictor=ToyModel(), 
    roi_size=(200, 200), # Infer on the whole image at once
    sw_batch_size=2, 
    overlap=0., 
    mode="constant")

px.imshow(output_tensor[0, 0])

px.imshow(output_tensor[1, 0])

px.imshow(output_tensor[2, 0])

notice: the value of the preds in the third image is 2, so given my batch_size and sw_batch_size, what has happened is that the first 2 inputs were considered, the model ran the inference on them (first step) and then the last input was considered and the model ran the inference on it (second step)... is it so?

Basically with roi_size I decide "how big" the patches taken from a single image are, while with "sw_batch_size" I decide "how many images to consider" while doing inference within a single step: can you confirm?

My doubt arises from the fact that when roi_size < input_size, the behaviour I just described seems to be applied not on the "batch dim", but rather on the "window dim", i.e.:

input_tensor = torch.zeros(3, 1, 200, 200)
output_tensor = sliding_window_inference(
    inputs=input_tensor, 
    predictor=ToyModel(), 
    roi_size=(50, 50), 
    sw_batch_size=2, 
    overlap=0., 
    mode="constant")

so, in the batch there are not windows from different images, but rather subsequent windows from the same input.
Is it so? Is it working this way because of memory efficiency?

I can see that this reasoning can be generalized to the one I made before, where the subwindows match exactly the original input and the batches become "traditional batches".

[KumoLiu]

Hi @sh3rlock14, when we use SlidingWindowInferer, we mainly consider when the original data is too large and the memory is not enough, at this time we can randomly crop during training and predict by sliding window during inference.
For sw_batch_size, you can simply treat it as batch dimension during inference, the total num of windows is windows_num_per_image * batch_size. Additionally, if your roi_size is equal to input_size you can use SimpleInferer directly instead of SlidingWindowInferer. For the overlap arg, it is mainly to ensure more accurate results of the boundary when doing sliding windows.

MONAI/monai/inferers/utils.py

Lines 147 to 149 in 4f584a9

	slices = dense_patch_slices(image_size, roi_size, scan_interval)
	num_win = len(slices) # number of windows per image
	total_slices = num_win * batch_size # total number of windows

Hope it can help you, thanks!