Spacingd with cuda type tensor is NOT working

[aitrics-chris]

According to https://docs.monai.io/en/latest/highlights.html , all the geometric image transforms (Spacing, Zoom, Rotate, Resize, etc.) support GPU acceleration via GPU Tensor operations for high performance.
However, I cannot use Spacingd after ToDeviced(keys=["image", "label"], device=f"cuda:0") within Compose.
Below is the error message occurred (from Spacingd()) when Spacingd comes after ToDeviced in Compose():

TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

I use DecathlonDataset(prostate) with Compose([LoadImaged, EnsureChannelFirstd, Orientationd, EnsureTyped, CropForegroundd, NormalizeIntensityd, ToDeviced, RandFlipd, Spacingd, RandCropByPosNegLabeld]). Thank you in advance!!! :)

[rijobro]

Hi @JoohyungLee0106, I've moved this discussion from the tutorials repo to here.

Could you give the full trace of the error (all the error message). It will help figure out which object is being unsuccessfully handled. Also which version of monai are you using? Our unit test checks that Spacingd works with CUDA torch.Tensors, but maybe we've missed something.

[rijobro]

Could you also give the full arguments to your transforms?

[aitrics-chris]

1) package version:

python: 3.7.11
monai: 0.7.0
pytorch: 1.9.1
cuda: 11.1.1
cudnn: 8.2.1
ubuntu 18.04.6 lts

2) full arguments of the transforms:

Compose([
LoadImaged(keys=["image", "label"]),
EnsureChannelFirstd(keys=["image", "label"]),
Orientationd(keys=["image", "label"], axcodes="RAS"),
EnsureTyped(keys=["image", "label"]),
CropForegroundd(keys=["image", "label"], source_key="image")
NormalizeIntensityd(keys="image", nonzero=False, channel_wise=True)
ToDeviced(keys=["image", "label"], device=f"cuda:0")
RandFlipd(keys=["image", "label"], prob=0.5, spatial_axis=0),
RandFlipd(keys=["image", "label"], prob=0.5, spatial_axis=1),
RandFlipd(keys=["image", "label"], prob=0.5, spatial_axis=2),
Spacingd(keys=["image", "label"], pixdim=[0.63,0.63,0.63], mode=("bilinear", "nearest")),
RandCropByPosNegLabeld(
            keys=["image", "label"],
            label_key="label",
            spatial_size=[192,192,8],
            pos=1.0,
            neg=1.0,
            num_samples=4
        )
])

3) full trace of the error:

=== Transform input info -- Spacingd ===
image statistics:
Type: <class 'torch.Tensor'>
Shape: torch.Size([2, 320, 320, 20])
Value range: (-1.1845402717590332, 5.000038146972656)
label statistics:
Type: <class 'torch.Tensor'>
Shape: torch.Size([1, 320, 320, 20])
Value range: (0.0, 2.0)
image_meta_dict statistics:
Type: <class 'dict'>
Value: {'sizeof_hdr': array(348, dtype=int32), 'extents': array(0, dtype=int32), 'session_error': array(0, dtype=int16), 'dim_info': array(0, dtype=uint8), 'dim': array([  4, 320, 320,  20,   2,   1,   1,   1], dtype=int16), 'intent_p1': array(0., dtype=float32), 'intent_p2': array(0., dtype=float32), 'intent_p3': array(0., dtype=float32), 'intent_code': array(0, dtype=int16), 'datatype': array(16, dtype=int16), 'bitpix': array(32, dtype=int16), 'slice_start': array(0, dtype=int16), 'pixdim': array([  1.     ,   0.625  ,   0.625  ,   3.6    , 170.38896,   0.     ,
         0.     ,   0.     ], dtype=float32), 'vox_offset': array(0., dtype=float32), 'scl_slope': array(nan, dtype=float32), 'scl_inter': array(nan, dtype=float32), 'slice_end': array(0, dtype=int16), 'slice_code': array(0, dtype=uint8), 'xyzt_units': array(10, dtype=uint8), 'cal_max': array(0., dtype=float32), 'cal_min': array(0., dtype=float32), 'slice_duration': array(0., dtype=float32), 'toffset': array(0., dtype=float32), 'glmax': array(0, dtype=int32), 'glmin': array(0, dtype=int32), 'qform_code': array(1, dtype=int16), 'sform_code': array(1, dtype=int16), 'quatern_b': array(0., dtype=float32), 'quatern_c': array(0., dtype=float32), 'quatern_d': array(0., dtype=float32), 'qoffset_x': array(-93.306, dtype=float32), 'qoffset_y': array(-114.5106, dtype=float32), 'qoffset_z': array(-84.9277, dtype=float32), 'srow_x': array([  0.625,   0.   ,   0.   , -93.306], dtype=float32), 'srow_y': array([   0.    ,    0.625 ,    0.    , -114.5106], dtype=float32), 'srow_z': array([  0.    ,   0.    ,   3.6   , -84.9277], dtype=float32), 'affine': array([[   0.625     ,    0.        ,    0.        ,  -93.30599976],
       [   0.        ,    0.625     ,    0.        , -114.51059723],
       [   0.        ,    0.        ,    3.5999999 ,  -84.92769623],
       [   0.        ,    0.        ,    0.        ,    1.        ]]), 'original_affine': array([[   0.625     ,    0.        ,    0.        ,  -93.30599976],
       [   0.        ,    0.625     ,    0.        , -114.51059723],
       [   0.        ,    0.        ,    3.5999999 ,  -84.92769623],
       [   0.        ,    0.        ,    0.        ,    1.        ]]), 'as_closest_canonical': False, 'spatial_shape': array([320, 320,  20], dtype=int16), 'original_channel_dim': -1, 'filename_or_obj': '/data2/data/monai/Task05_Prostate/imagesTr/prostate_43.nii.gz'}
label_meta_dict statistics:
Type: <class 'dict'>
Value: {'sizeof_hdr': array(348, dtype=int32), 'extents': array(0, dtype=int32), 'session_error': array(0, dtype=int16), 'dim_info': array(0, dtype=uint8), 'dim': array([  3, 320, 320,  20,   1,   1,   1,   1], dtype=int16), 'intent_p1': array(0., dtype=float32), 'intent_p2': array(0., dtype=float32), 'intent_p3': array(0., dtype=float32), 'intent_code': array(0, dtype=int16), 'datatype': array(2, dtype=int16), 'bitpix': array(8, dtype=int16), 'slice_start': array(0, dtype=int16), 'pixdim': array([1.   , 0.625, 0.625, 3.6  , 0.   , 0.   , 0.   , 0.   ],
      dtype=float32), 'vox_offset': array(0., dtype=float32), 'scl_slope': array(nan, dtype=float32), 'scl_inter': array(nan, dtype=float32), 'slice_end': array(0, dtype=int16), 'slice_code': array(0, dtype=uint8), 'xyzt_units': array(10, dtype=uint8), 'cal_max': array(0., dtype=float32), 'cal_min': array(0., dtype=float32), 'slice_duration': array(0., dtype=float32), 'toffset': array(0., dtype=float32), 'glmax': array(0, dtype=int32), 'glmin': array(0, dtype=int32), 'qform_code': array(1, dtype=int16), 'sform_code': array(1, dtype=int16), 'quatern_b': array(0., dtype=float32), 'quatern_c': array(0., dtype=float32), 'quatern_d': array(0., dtype=float32), 'qoffset_x': array(-93.306, dtype=float32), 'qoffset_y': array(-114.5106, dtype=float32), 'qoffset_z': array(-84.9277, dtype=float32), 'srow_x': array([  0.625,   0.   ,   0.   , -93.306], dtype=float32), 'srow_y': array([   0.    ,    0.625 ,    0.    , -114.5106], dtype=float32), 'srow_z': array([  0.    ,   0.    ,   3.6   , -84.9277], dtype=float32), 'affine': array([[   0.625     ,    0.        ,    0.        ,  -93.30599976],
       [   0.        ,    0.625     ,    0.        , -114.51059723],
       [   0.        ,    0.        ,    3.5999999 ,  -84.92769623],
       [   0.        ,    0.        ,    0.        ,    1.        ]]), 'original_affine': array([[   0.625     ,    0.        ,    0.        ,  -93.30599976],
       [   0.        ,    0.625     ,    0.        , -114.51059723],
       [   0.        ,    0.        ,    3.5999999 ,  -84.92769623],
       [   0.        ,    0.        ,    0.        ,    1.        ]]), 'as_closest_canonical': False, 'spatial_shape': array([320, 320,  20], dtype=int16), 'original_channel_dim': 'no_channel', 'filename_or_obj': '/data2/data/monai/Task05_Prostate/labelsTr/prostate_43.nii.gz'}
image_transforms statistics:
Type: <class 'list'>
Value: [{'class': 'Orientationd', 'id': 140164871824528, 'orig_size': (320, 320, 20), 'extra_info': {'meta_key': 'image_meta_dict', 'old_affine': array([[   0.625     ,    0.        ,    0.        ,  -93.30599976],
       [   0.        ,    0.625     ,    0.        , -114.51059723],
       [   0.        ,    0.        ,    3.5999999 ,  -84.92769623],
       [   0.        ,    0.        ,    0.        ,    1.        ]])}}, {'class': 'EnsureTyped', 'id': 140164871824656, 'orig_size': (320, 320, 20)}, {'class': 'CropForegroundd', 'id': 140164871810576, 'orig_size': torch.Size([320, 320, 20]), 'extra_info': {'box_start': array([0, 0, 0]), 'box_end': array([320, 320,  20])}}, {'class': 'RandFlipd', 'id': 140164871824464, 'orig_size': torch.Size([320, 320, 20]), 'do_transforms': True}, {'class': 'RandFlipd', 'id': 140164871824976, 'orig_size': torch.Size([320, 320, 20]), 'do_transforms': True}, {'class': 'RandFlipd', 'id': 140164871825104, 'orig_size': torch.Size([320, 320, 20]), 'do_transforms': False}]
label_transforms statistics:
Type: <class 'list'>
Value: [{'class': 'Orientationd', 'id': 140164871824528, 'orig_size': (320, 320, 20), 'extra_info': {'meta_key': 'label_meta_dict', 'old_affine': array([[   0.625     ,    0.        ,    0.        ,  -93.30599976],
       [   0.        ,    0.625     ,    0.        , -114.51059723],
       [   0.        ,    0.        ,    3.5999999 ,  -84.92769623],
       [   0.        ,    0.        ,    0.        ,    1.        ]])}}, {'class': 'EnsureTyped', 'id': 140164871824656, 'orig_size': (320, 320, 20)}, {'class': 'CropForegroundd', 'id': 140164871810576, 'orig_size': torch.Size([320, 320, 20]), 'extra_info': {'box_start': array([0, 0, 0]), 'box_end': array([320, 320,  20])}}, {'class': 'RandFlipd', 'id': 140164871824464, 'orig_size': torch.Size([320, 320, 20]), 'do_transforms': True}, {'class': 'RandFlipd', 'id': 140164871824976, 'orig_size': torch.Size([320, 320, 20]), 'do_transforms': True}, {'class': 'RandFlipd', 'id': 140164871825104, 'orig_size': torch.Size([320, 320, 20]), 'do_transforms': False}]
foreground_start_coord statistics:
Type: <class 'numpy.ndarray'>
Shape: (3,)
Value range: (0, 0)
foreground_end_coord statistics:
Type: <class 'numpy.ndarray'>
Shape: (3,)
Value range: (20, 320)
Exception in thread Thread-10:
Traceback (most recent call last):
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/transforms/transform.py", line 92, in apply_transform
    return _apply_transform(transform, data, unpack_items)
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/transforms/transform.py", line 60, in _apply_transform
    return transform(parameters)
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/transforms/spatial/dictionary.py", line 229, in __call__
    data_array=np.asarray(d[key]),
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/numpy/core/_asarray.py", line 102, in asarray
    return array(a, dtype, copy=False, order=order)
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/torch/_tensor.py", line 643, in __array__
    return self.numpy()
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/anaconda3/envs/monai/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/home/user/anaconda3/envs/monai/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/data/thread_buffer.py", line 45, in enqueue_values
    for src_val in self.src:
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 561, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/data/dataset.py", line 95, in __getitem__
    return self._transform(index)
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/data/dataset.py", line 667, in _transform
    data = apply_transform(_transform, data)
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/transforms/transform.py", line 116, in apply_transform
    raise RuntimeError(f"applying transform {transform}") from e
RuntimeError: applying transform <monai.transforms.spatial.dictionary.Spacingd object at 0x7f7aad652750>

Traceback (most recent call last):
  File "/home/user/Codes_Research/AnisotropicMedicalImage/speed_fast_novo.py", line 209, in <module>
    epoch_loss /= i
NameError: name 'i' is not defined

Process finished with exit code 1

[rijobro]

Thanks. Could you do something similar to the following for me?

data = {"image": some_filename, "label": some_filename}
transform = Compose([
    LoadImaged(keys=["image", "label"]),
    EnsureChannelFirstd(keys=["image", "label"]),
    EnsureTyped(keys=["image", "label"]),
    ToDeviced(keys=["image", "label"], device=f"cuda:0"),
    Spacingd(keys=["image", "label"], pixdim=[0.63,0.63,0.63], mode=("bilinear", "nearest")),
])
transform(data)

And tell me what the error given then is?

[aitrics-chris]

Sure!!

=== Transform input info -- Spacingd ===
image statistics:
Type: <class 'torch.Tensor'>
Shape: torch.Size([2, 320, 320, 20])
Value range: (0.0, 2754.0)
label statistics:
Type: <class 'torch.Tensor'>
Shape: torch.Size([1, 320, 320, 20])
Value range: (0.0, 2.0)
image_meta_dict statistics:
Type: <class 'dict'>
Value: {'sizeof_hdr': array(348, dtype=int32), 'extents': array(0, dtype=int32), 'session_error': array(0, dtype=int16), 'dim_info': array(0, dtype=uint8), 'dim': array([  4, 320, 320,  20,   2,   1,   1,   1], dtype=int16), 'intent_p1': array(0., dtype=float32), 'intent_p2': array(0., dtype=float32), 'intent_p3': array(0., dtype=float32), 'intent_code': array(0, dtype=int16), 'datatype': array(16, dtype=int16), 'bitpix': array(32, dtype=int16), 'slice_start': array(0, dtype=int16), 'pixdim': array([  1.       ,   0.6249998,   0.625    ,   3.5999987, 153.34152  ,
         0.       ,   0.       ,   0.       ], dtype=float32), 'vox_offset': array(0., dtype=float32), 'scl_slope': array(nan, dtype=float32), 'scl_inter': array(nan, dtype=float32), 'slice_end': array(0, dtype=int16), 'slice_code': array(0, dtype=uint8), 'xyzt_units': array(10, dtype=uint8), 'cal_max': array(0., dtype=float32), 'cal_min': array(0., dtype=float32), 'slice_duration': array(0., dtype=float32), 'toffset': array(0., dtype=float32), 'glmax': array(0, dtype=int32), 'glmin': array(0, dtype=int32), 'qform_code': array(1, dtype=int16), 'sform_code': array(1, dtype=int16), 'quatern_b': array(1.3442837e-24, dtype=float32), 'quatern_c': array(-0.0130896, dtype=float32), 'quatern_d': array(1.540876e-33, dtype=float32), 'qoffset_x': array(-87.50963, dtype=float32), 'qoffset_y': array(-108.7274, dtype=float32), 'qoffset_z': array(-63.50623, dtype=float32), 'srow_x': array([ 6.2478566e-01, -2.1995171e-26, -9.4237015e-02, -8.7509628e+01],
      dtype=float32), 'srow_y': array([-2.1995163e-26,  6.2500000e-01, -9.6780099e-24, -1.0872740e+02],
      dtype=float32), 'srow_z': array([ 1.6360594e-02,  1.6802106e-24,  3.5987651e+00, -6.3506229e+01],
      dtype=float32), 'affine': array([[ 6.24785662e-01, -2.19951708e-26, -9.42370147e-02,
        -8.75096283e+01],
       [-2.19951630e-26,  6.25000000e-01, -9.67800985e-24,
        -1.08727402e+02],
       [ 1.63605940e-02,  1.68021062e-24,  3.59876513e+00,
        -6.35062294e+01],
       [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         1.00000000e+00]]), 'original_affine': array([[ 6.24785662e-01, -2.19951708e-26, -9.42370147e-02,
        -8.75096283e+01],
       [-2.19951630e-26,  6.25000000e-01, -9.67800985e-24,
        -1.08727402e+02],
       [ 1.63605940e-02,  1.68021062e-24,  3.59876513e+00,
        -6.35062294e+01],
       [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         1.00000000e+00]]), 'as_closest_canonical': False, 'spatial_shape': array([320, 320,  20], dtype=int16), 'original_channel_dim': -1, 'filename_or_obj': '/data2/data/monai/Task05_Prostate/imagesTr/prostate_31.nii.gz'}
label_meta_dict statistics:
Type: <class 'dict'>
Value: {'sizeof_hdr': array(348, dtype=int32), 'extents': array(0, dtype=int32), 'session_error': array(0, dtype=int16), 'dim_info': array(0, dtype=uint8), 'dim': array([  3, 320, 320,  20,   1,   1,   1,   1], dtype=int16), 'intent_p1': array(0., dtype=float32), 'intent_p2': array(0., dtype=float32), 'intent_p3': array(0., dtype=float32), 'intent_code': array(0, dtype=int16), 'datatype': array(2, dtype=int16), 'bitpix': array(8, dtype=int16), 'slice_start': array(0, dtype=int16), 'pixdim': array([1.        , 0.62499976, 0.625     , 3.5999987 , 0.        ,
       0.        , 0.        , 0.        ], dtype=float32), 'vox_offset': array(0., dtype=float32), 'scl_slope': array(nan, dtype=float32), 'scl_inter': array(nan, dtype=float32), 'slice_end': array(0, dtype=int16), 'slice_code': array(0, dtype=uint8), 'xyzt_units': array(10, dtype=uint8), 'cal_max': array(0., dtype=float32), 'cal_min': array(0., dtype=float32), 'slice_duration': array(0., dtype=float32), 'toffset': array(0., dtype=float32), 'glmax': array(0, dtype=int32), 'glmin': array(0, dtype=int32), 'qform_code': array(1, dtype=int16), 'sform_code': array(1, dtype=int16), 'quatern_b': array(1.3442837e-24, dtype=float32), 'quatern_c': array(-0.0130896, dtype=float32), 'quatern_d': array(1.540876e-33, dtype=float32), 'qoffset_x': array(-87.50961, dtype=float32), 'qoffset_y': array(-108.7274, dtype=float32), 'qoffset_z': array(-63.50623, dtype=float32), 'srow_x': array([ 6.2478560e-01, -2.1995171e-26, -9.4237015e-02, -8.7509613e+01],
      dtype=float32), 'srow_y': array([-2.1995162e-26,  6.2500000e-01, -9.6780099e-24, -1.0872740e+02],
      dtype=float32), 'srow_z': array([ 1.6360592e-02,  1.6802106e-24,  3.5987651e+00, -6.3506229e+01],
      dtype=float32), 'affine': array([[ 6.24785602e-01, -2.19951708e-26, -9.42370147e-02,
        -8.75096130e+01],
       [-2.19951615e-26,  6.25000000e-01, -9.67800985e-24,
        -1.08727402e+02],
       [ 1.63605921e-02,  1.68021062e-24,  3.59876513e+00,
        -6.35062294e+01],
       [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         1.00000000e+00]]), 'original_affine': array([[ 6.24785602e-01, -2.19951708e-26, -9.42370147e-02,
        -8.75096130e+01],
       [-2.19951615e-26,  6.25000000e-01, -9.67800985e-24,
        -1.08727402e+02],
       [ 1.63605921e-02,  1.68021062e-24,  3.59876513e+00,
        -6.35062294e+01],
       [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         1.00000000e+00]]), 'as_closest_canonical': False, 'spatial_shape': array([320, 320,  20], dtype=int16), 'original_channel_dim': 'no_channel', 'filename_or_obj': '/data2/data/monai/Task05_Prostate/labelsTr/prostate_31.nii.gz'}
image_transforms statistics:
Type: <class 'list'>
Value: [{'class': 'EnsureTyped', 'id': 139904233648592, 'orig_size': (320, 320, 20)}]
label_transforms statistics:
Type: <class 'list'>
Value: [{'class': 'EnsureTyped', 'id': 139904233648592, 'orig_size': (320, 320, 20)}]
Loading dataset:   0%|          | 0/26 [00:02<?, ?it/s]
Modifying image pixdim from [0.625   0.625   3.60001 1.     ] to [  0.625        0.625        3.60000992 151.21210251]
Traceback (most recent call last):
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/transforms/transform.py", line 92, in apply_transform
    return _apply_transform(transform, data, unpack_items)
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/transforms/transform.py", line 60, in _apply_transform
    return transform(parameters)
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/transforms/spatial/dictionary.py", line 229, in __call__
    data_array=np.asarray(d[key]),
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/numpy/core/_asarray.py", line 102, in asarray
    return array(a, dtype, copy=False, order=order)
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/torch/_tensor.py", line 643, in __array__
    return self.numpy()
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/Codes_Research/AnisotropicMedicalImage/speed_fast_novo.py", line 141, in <module>
    download=True, seed=args.seed, num_workers=0, cache_rate=1.0, val_frac=0.2)
  File "/home/user/Codes_Research/AnisotropicMedicalImage/utils/decathlon_cache.py", line 92, in __init__
    self, data, transform, cache_num=cache_num, cache_rate=cache_rate, num_workers=num_workers
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/data/dataset.py", line 605, in __init__
    self._cache: List = self._fill_cache()
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/data/dataset.py", line 630, in _fill_cache
    desc="Loading dataset",
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/tqdm/std.py", line 1180, in __iter__
    for obj in iterable:
  File "/home/user/anaconda3/envs/monai/lib/python3.7/multiprocessing/pool.py", line 748, in next
    raise value
  File "/home/user/anaconda3/envs/monai/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/data/dataset.py", line 646, in _load_cache_item
    item = apply_transform(_xform, item)
  File "/home/user/anaconda3/envs/monai/lib/python3.7/site-packages/monai/transforms/transform.py", line 116, in apply_transform
    raise RuntimeError(f"applying transform {transform}") from e
RuntimeError: applying transform <monai.transforms.spatial.dictionary.Spacingd object at 0x7f3dfe263310>

Process finished with exit code 1

[Nic-Ma]

Hi @JoohyungLee0106 ,

May I know your MONAI version?
I feel maybe you are not using the latest dev branch, Spacing transform just added GPU Tensor support recently after MONAI v0.7 release.

Thanks.

[aitrics-chris]

As I have stated above, my monai version is 0.7.0. Will that be a problem?

[Nic-Ma]

OK, to use GPU Tensor for Spacing transform, please try to use the latest dev code or at least nightly build.

Thanks.

[aitrics-chris]

1. I have upgrade monai via pip install monai --upgrade
   => Spacingd still doesn't work with cuda tensor.

2. I have installed Weekly preview release via pip install monai-weekly
   => Spacingd DOES work with cuda tensor.

Thank you for the suggestion, @Nic-Ma and @rijobro !!!