error trying to inference on numpy data without ever converting to nifti

[AceMcAwesome77]

Hi, I have a trained RetinaNet3D MONAI model and am trying to set up an inference pipeline in which I never actually have to create a nifti file. I start with my data in a 3D numpy array so in theory this should be possible. In fact, I am able to get the correct image tensor shape and values by doing this:

device = 'cuda'

data = np.expand_dims(data_zoom, axis=0)
data = torch.from_numpy(data)
data = [data.to(device)]

where data_zoom is my starting 3D numpy array. I then run this for inference:

with torch.no_grad():
        with torch.cuda.amp.autocast():
                inference_outputs = detector(data, use_inferer=True)

but unfortunately get this error:

Traceback (most recent call last):
  Cell In[6], line 3
    inference_outputs = detector(data, use_inferer=True)
  File ~\AppData\Local\anaconda3\lib\site-packages\torch\nn\modules\module.py:1501 in _call_impl
    return forward_call(*args, **kwargs)
  File ~\AppData\Local\anaconda3\lib\site-packages\monai\apps\detection\networks\retinanet_detector.py:495 in forward
    head_outputs = predict_with_inferer(
  File ~\AppData\Local\anaconda3\lib\site-packages\monai\apps\detection\utils\predict_utils.py:130 in predict_with_inferer
    head_outputs_sequence = inferer(images, _network_sequence_output, network, keys=keys)
  File ~\AppData\Local\anaconda3\lib\site-packages\monai\inferers\inferer.py:202 in __call__
    return sliding_window_inference(
  File ~\AppData\Local\anaconda3\lib\site-packages\monai\inferers\utils.py:167 in sliding_window_inference
    min_non_zero = max(importance_map_[importance_map_ != 0].min().item(), 1e-3)
RuntimeError: min(): Expected reduction dim to be specified for input.numel() == 0. Specify the reduction dim with the 'dim' argument.

Note that my code works and inferences properly if I use the nifti and DataLoader module instead, as is done in the tutorial. The following code works properly:

test_transforms = Compose(
    [
        LoadImaged(keys=["image"]),
        EnsureChannelFirstd(keys=["image"]),
        EnsureTyped(keys=["image"], dtype=torch.float32), # pylint: disable=E1101
        Orientationd(keys=["image"], axcodes="RAS"),
        EnsureTyped(keys=["image"], dtype=torch.float32), # pylint: disable=E1101
    ]
)

data_nifti = nib.Nifti1Image(data_zoom, affine=np.eye(4))
data_save_path = os.path.join(nifti_folder, os.path.basename(folder) + '.nii.gz')
nib.save(data_nifti, data_save_path)

inference_data = [{'image': data_save_path, 'label': [0]}]
                
inference_ds = Dataset(
     data=inference_data,
     transform=test_transforms
                )
                
inference_loader = DataLoader(
            inference_ds,
            batch_size=1,
            num_workers=0,
            pin_memory=torch.cuda.is_available(),
            collate_fn=no_collation,
                )

iterator = iter(inference_loader)
inference_data = next(iterator)
inference_inputs = [inference_data[0]["image"].to(device)]

with torch.no_grad():
        with torch.cuda.amp.autocast():
                  inference_outputs = detector(inference_inputs, use_inferer=True)

Fascinatingly, the input tensors are equivalent despite one causing the error and the other working. For both the following:

inference_inputs[0].shape
data[0].shape

I get the following:

torch.Size([1, 356, 356, 196])

and this equation returns TRUE:

torch.equal(inference_inputs[0], data[0])

One difference I do see with these tensors is that the type of data[0] is torch.Tensor and the type of inference_inputs[0] is monai.data.meta_tensor.MetaTensor.

Does anyone know what is causing this error, or if there is a better way to go directly from a numpy array to inferencing without having to convert to nifti format in between? Thanks!

[KumoLiu]

Hi @AceMcAwesome77, I didn't read them all, but my assumption is that the absence of a batch dimension is the cause of your error. To check, you can simply add a dimension in front of the data.
Hope it can help you, thanks!

[AceMcAwesome77]

Thanks for the reply - but if that was the case, how could the tensors between my working code and broken code be equal then? See that last part about the torch.equal check.

[AceMcAwesome77]

I just solved this actually - the problem was that the tensor in data[0] was dtype uint8 and the working tensor was dtype float32. Even though they are different datatypes the tensors were showing up as equal because they had the same values. My problem was solved by adding this simple .float() line:

data = np.expand_dims(data_zoom, axis=0)
data = torch.from_numpy(data)
data = data.float()
data = [data.to(device)]