Sequential spacing does not work

[BardiaKh]

Describe the bug
Suppose that I have an image and when the image is loaded it will be converted to a high-resolution format using:

transforms=mn.transforms.Compose([
    mn.transforms.LoadImageD(keys=["image"],reader="NibabelReader"),
    mn.transforms.SpacingD(keys=["image"], pixdim=[1,1,1], padding_mode=["zeros"], mode=["bilinear"]),
    mn.transforms.ToTensorD(keys=["image"], dtype=torch.float),
    mn.transforms.ToMetaTensorD(keys=["image"])
])

dataset = mn.data.Dataset(data=data_dict, transform=transforms)

Then in another part of the pipeline, I want to convert it to a low-resolution image so I would say:

low_res_spacing = mn.transforms.SpacingD(kyes=["image"], pixdim=[2,2,2], padding_mode="zeros", mode="bilinear")
sample = dataset[0]
low_res_image = low_res_spacing(sample)

This should work fine, but the spacing is wrong and the image is very low res:

I add the low_res_spacing transform to the compose pipeline it works fine. As soon as MetaTensor is out of the compose, the spacing transform fails to work properly:

Because the MetaTensor has the affine information, the transform should work properly. And if I don't convert it to MetaTensor, it says:

`data_array` is not of type MetaTensor, assuming affine to be identity.

Which is not.

Expected behavior
I suppose the user should be able to apply spacing transforms after applying a composed pipeline of transforms.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment

Ensuring you use the relevant python executable, please paste the output of:

================================
Printing MONAI config...
================================
MONAI version: 0.9.1
Numpy version: 1.22.4
Pytorch version: 1.10.1
MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
MONAI rev id: 356d2d2f41b473f588899d705bbc682308cee52c
MONAI __file__: /home/m253231/anaconda3/envs/PyTorchDefault/lib/python3.9/site-packages/monai/__init__.py
Optional dependencies:
Pytorch Ignite version: NOT INSTALLED or UNKNOWN VERSION.
Nibabel version: 3.2.1
scikit-image version: 0.18.2
Pillow version: 8.3.1
Tensorboard version: 2.6.0
gdown version: NOT INSTALLED or UNKNOWN VERSION.
TorchVision version: 0.11.2
tqdm version: 4.62.2
lmdb version: NOT INSTALLED or UNKNOWN VERSION.
psutil version: 5.8.0
pandas version: 1.3.5
einops version: 0.3.2
transformers version: NOT INSTALLED or UNKNOWN VERSION.
mlflow version: NOT INSTALLED or UNKNOWN VERSION.
pynrrd version: NOT INSTALLED or UNKNOWN VERSION.

For details about installing the optional dependencies, please visit:
    https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

[BardiaKh]

Okay, playing a little bit more with it show that if the Compose pipeline changes to the following it works fine:

transforms=mn.transforms.Compose([
    mn.transforms.LoadImageD(keys=["image"],reader="NibabelReader"),
    mn.transforms.ToMetaTensorD(keys=["image"])
    mn.transforms.SpacingD(keys=["image"], pixdim=[1,1,1], padding_mode=["zeros"], mode=["bilinear"]),
    mn.transforms.ToTensorD(keys=["image"], dtype=torch.float),
])

i.e., first converting to MetaTensor and then doing all the spacings. But this is not a good user experience, and I believe MetaTensor should get the affine information from previous transforms, regardless of order.

[wyli]

Thanks, it's the ToTensorD that drops all the meta info and causes the spacing issues. I'd suggest removing all the ToTensorD in the pipeline and then retrying the runs.

[BardiaKh]

That works as you said. So all instances of ToTensor should be changed to ToMetaTensor, right? If so, is there a way to set the dtype in ToMetaTensor?

[wyli]

That's great and sorry I didn't know that ToTensor is used in your example to change the dtypes. If that's the case you can keep them and set track_meta=True in the transform (the default track_meta is False in monai v0.9.1 but it will be improved in the upcoming release). dtype=torch.float, track_meta=True will preserve the meta info and change the dtype.

ToMetaTensorD is for converting a dictionary {'img':..., 'img_meta_dict':...} into a metatensor, mainly for compatibility with the previous version of monai. so it's probably not needed in the pipeline.

btw LoadImageD by default outputs MetaTensors, please see examples in https://github.com/Project-MONAI/tutorials/tree/main/3d_segmentation