Select only 2D slices that contain label from 3D volume

[valecorbetta]

Hi,
I'm trying to use a 2D UNet to segment brain lesions (I know that a 3D would be better but I need it to be 2D). Since the lesions are present in just a few slices and the occupy a small portion of these slices, I would like to select only the slices in which the label is present. Also, I don't want the cropped sample to be centered around the slices. My input volumes have differnt number of slices.
I have tried to do the following:

train_transforms = Compose ( [ LoadImaged(keys=keys), AddChanneld(keys=keys), NormalizeIntensityd(keys=image_keys, nonzero=True), #nonzero=True -> normalize only non-zero values RandCropByPosNegLabeld(keys=keys, label_key="label", image_key="image", spatial_size=(128,128,128), num_samples=32, pos=4, neg=1), RandSpatialCropd(keys=keys, roi_size=(96,96,96), random_center=True, random_size=False), RandSpatialCropSamplesd(keys=keys, roi_size=(96,96,1), num_samples=96, random_center=True, random_size=False), ToTensord(keys=keys) ] )

But I encounter the following error:

ValueError: dictionary update sequence element #0 has length 4; 2 is required

So I'm guessing I'm doing something wrong. Does anyone have any suggestions on how to achieve this?

Thanks

[KumoLiu]

Hi @valecorbetta, I noticed that you used num_samples in both RandCropByPosNegLabeld and RandSpatialCropSamplesd. After your transforms, it will create a nested list that contains 32*96 samples for each image.
IMO, you could simply remove RandSpatialCropSamplesd and changed the roi_size in RandSpatialCropd to be (96, 96, 1).

Hope it can help you, thanks!

[KumoLiu]

Hi @valecorbetta, since you set the num_samples equal to 96, do you want all these slices?
I'm not sure why you need to crop several times. Since all these 32*96 samples are selected from only one image. If you just want to select the 2d slices containing the label, you could use RandCropByPosNegLabeld(keys=keys, label_key="label", image_key="image", spatial_size=(96, 96, 1), num_samples=32*96, pos=4, neg=0). Does this not satisfy your needs?

[naga-karthik]

Hey @KumoLiu, I also have a similar requirement of being able to select only those slices with labels. As per your suggestion, if we use RandCropByPosNegLabeld(keys=keys, label_key="label", image_key="image", spatial_size=(96, 96, 1), num_samples=4, pos=4, neg=0), am I correct in understanding that this transformation selects 4 slices per image that only have the labels (at least this is what I understand by setting neg=0)? What happens to the rest of the slices in the image that doesn't have any labels?

Also, this is different from selecting slices that definitely have the labels right and discarding the rest of the image, right? I mean, doesn't this transformation raise a warning for slices that don't have any labels?

I guess I am getting confused with how exactly this RandCropByPosNegLabeld function works for inputs with a lot less labels (for e.g. lesions) than the typical images. Any help is greatly appreciated!

[KumoLiu]

Hi @naga-karthik, RandCropByPosNegLabel crop regions with the center being a foreground or background voxel based on the Pos Neg Ratio, so neg=0 means all 4 slices will be cropped with foreground as the center.
If you set neg=0, the slices selected will definitely have the foreground, it will also give a warning if no foreground can be centered.

MONAI/monai/transforms/utils.py

Lines 509 to 517 in a8302ec

	if len(fg_indices) == 0 and len(bg_indices) == 0:
	raise ValueError("No sampling location available.")
	if len(fg_indices) == 0 or len(bg_indices) == 0:
	pos_ratio = 0 if len(fg_indices) == 0 else 1
	warnings.warn(
	f"Num foregrounds {len(fg_indices)}, Num backgrounds {len(bg_indices)}, "
	f"unable to generate class balanced samples, setting `pos_ratio` to {pos_ratio}."
	)

Hope it can help you, Thanks!

[naga-karthik]

Ah okay, this helps! Thanks a lot for clarifying!

[valecorbetta]

Hi, sorry, I was convinced I had replied to the thread, @KumoLiu your solution worked really well. Thanks again (and sorry for the late response)

[jcohenadad]

Hi, I am also trying to select only slices that have non-empty labels. When trying the code you proposed it works perfectly well (ie: only non-empty label slices are selected):

train_transforms = Compose(
    [
        LoadImaged(keys=["image", "label"]),
        EnsureChannelFirstd(keys=["image", "label"]),
        ScaleIntensityRanged(keys=["image"], a_min=0, a_max=1, b_min=0.0, b_max=1.0, clip=True),
        Spacingd(keys=["image", "label"], pixdim=(0.05, 0.05, 0.05), mode=("bilinear", "nearest")),
        RandCropByPosNegLabeld(
            keys=["image", "label"],
            image_key="image",
            label_key="label",
            spatial_size=(200, 200, 1),
            pos=1,
            neg=0,
            num_samples=1,
        ),
    ]
)

However, when attempting the same thing using a different leading method as per this tutorial with the monai.data.PatchIterd and monai.data.GridPatchDataset functions, all slices are selected (incl. those without labels in them):

# Volume-level transforms for both image and segmentation
train_transforms = Compose(
    [
        LoadImaged(keys=["image", "label"]),
        EnsureChannelFirstd(keys=["image", "label"]),
        ScaleIntensityd(keys="image"),
        EnsureTyped(keys=["image", "label"]),
    ]
)
# 3D dataset with preprocessing transforms
volume_ds = CacheDataset(data=train_files, transform=train_transforms)

# Volume to patch processing
patch_func = PatchIterd(
    keys=["image", "label"], patch_size=(None, None, 1), start_pos=(0, 0, 0)  # dynamic first two dimensions
)
patch_transform = Compose(
    [
        SqueezeDimd(keys=["image", "label"], dim=-1),
        RandCropByPosNegLabeld(
            keys=["image", "label"],
            label_key="label",
            spatial_size=(200, 200),
            pos=1,
            neg=0,
            num_samples=1, 
        ),
    ]
)
patch_ds = GridPatchDataset(
    data=volume_ds, patch_iter=patch_func, transform=patch_transform, with_coordinates=False
)
shuffle_ds = ShuffleBuffer(patch_ds, buffer_size=30, seed=0)
train_loader = DataLoader(patch_ds,
                          batch_size=config['train_batch_size'],
                          num_workers=config['num_workers'],
                          pin_memory=torch.cuda.is_available())

I cannot wrap my head around what I am doing wrong in the 2nd scenario. Also, related question: what are the benefits of going with the monai.data.PatchIterd and monai.data.GridPatchDataset approach?

[KumoLiu]

Could you show the whole error message? Thanks!

[KumoLiu]

Just find your source code, I think you should use EnsureChannelFirstd(keys=["image", "label"]) instead of EnsureChannelFirst(). Thanks!

[jcohenadad]

Good catch! The issue is indeed now solved. Thank you! While continuing to investigate the parallel approach "A" described in #5948 (reply in thread) (latest working prototype at https://github.com/ivadomed/model_seg_mouse-sc_wm-gm_t1/blob/main/train.py), I can try the approach "B" proposed by @KumoLiu in https://github.com/ivadomed/model_seg_mouse-sc_wm-gm_t1/blob/jca/loader-test/train.py. But given that in this particular application, I would like to mix train/validation volumes, is there an actual incentive to work with "B" vs. "A"?

[KumoLiu]

Didn't get your questions, did you mean PatchDataset and GridPatchDataset?

[jcohenadad]

Sorry for not being clear. Here are my questions:

• With this approach, the split is done on the 3D volumes, which in this case is undesired. Instead, I would like to extract all non-empty 2D slices across all volumes, and then do the train/validation split. Is there a way to split a "DataLoader()" object?
• There is no fixed number of non-empty 2D slices per 3D volume (eg: volume 1 has 11, volume 2 has 8, etc.), therefore, setting samples_per_image to a fixed number would not work. Is there a dynamic way to set samples_per_image for each volume?
  Thanks!