Using different window level(intensity range) as multichannel input to UNET architecture.

[ArpanGyawali]

My task: Skull Stripping/ Brain Extraction from head CTA.

What i have already done:

• To change the window width level to (200, 600), i applied the following preprocessing step
  ScaleIntensityRanged(keys="image", a_min=-100, a_max=500, b_min=0.0, b_max=1.0, clip=True)
  the prediction result from my model are good but still not satisfactory

My concern:

• To improve the accuracy, as an experimentation, I am trying to use 3 different window level for training as 3 channel input to UNET architecture. The original image will be the same but each input channel needs to have different window(intensity range). How can i do this?

My thoughts:
In the network: use in-channel: 3

• option1: load a single image and add 3 different value for ScaleIntensityRanged transform and pass them as different channel to the network. I don't know how to do that or if that's possible
• option2: load 3 different image with the different window and exclude ScaleIntensityRanged transform from preprocessing step.

Can i have some intuition and help on this. Thanks on advance.

[KumoLiu]

Hi @ArpanGyawali,

You can use ConcatItemsd to concat different images into a big array.

MONAI/monai/transforms/utility/dictionary.py

Line 927 in 95f69de

class ConcatItemsd(MapTransform):

Hope it helps, thanks.

[KumoLiu]

1. Yes, you should be having in_channels=4 in your network architecture definition, given that you're concatenating four images along the channel axis.
2. While it's true that loading four different images for each sample could consume more memory, this additional resource consumption depends on the specifics of the pipeline. One thing you can do to save memory is to use torch.float16 instead of torch.float32 data type for images, if that's acceptable for your use case.
3. It's possible to perform the windowing independently on the same initial image stored in memory and then concatenate these results. MONAI's ConcatItemsd transform can handle this. You can use Lambdad to save four different keys for different images.
   Here's an example:

# Initial transforms for loading data
transforms = Compose([
    LoadImaged(keys='image', ensure_channel_first),
    Spacingd(keys='image', pixdim=(1.5, 1.5, 2.0), mode='bilinear')
])

windows = [
    {"a_min": -175, "a_max": 275},
    {"a_min": -100, "a_max": 200},
    {"a_min": -200, "a_max": 300},
    {"a_min": -150, "a_max": 250},
]

new_keys = ["image1", "image2", "image3", "image4"]

windowing_transforms = [
    Lambdad(keys, ScaleIntensityRanged, {"a_min": w["a_min"], "a_max": w["a_max"], "b_min":0.0, "b_max":1.0, "clip":True})
    for w in windows
]

final_trans = Compose(transforms + windowing_transforms)

In this example, you would be executing a list of identical operations but with different parameters on the same data point, and concatenate the results. In this case, the output of each ScaleIntensityRanged transform would be saved and then concatenated.

Hope it helps, thanks.

[ArpanGyawali]

Thank you so much. This really helped me.

[ArpanGyawali]

@KumoLiu
Hi I discontinued that project for a while and i am back now.
Here whats the use of "new_keys". It not used in the lambda function. Are the keys for each channel different.
Then how will we apply remaining other preprocessing and augmentation on the new_keys?

[KumoLiu]

For the remaining dictionary version transform, you can just add new_keys to keys in the transforms.
Thanks.

[ArpanGyawali]

@KumoLiu
Shouldnt the line be
windowing_transforms = [
Lambdad(keys, ScaleIntensityRanged, {'image', "a_min": w["a_min"], "a_max": w["a_max"], "b_min":0.0, "b_max":1.0, "clip":True})
for w, keys in zip(windows, new_keys)
]

[ArpanGyawali]

Hey, @KumoLiu
I used the Lmabdad function in this way:

def train_pre_transforms(self, context: Context):
        # Initial transforms
        transforms = [
            LoadImaged(keys=("image", "label")),
            EnsureChannelFirstd(keys=("image", "label")),
            Spacingd(
                keys=("image", "label"),
                pixdim=(1.0, 1.0, 1.0),
                mode=("bilinear", "nearest"),
                ),
        ]

        # Define the intensity range for each channel
        windows = [
            {"a_min": -50, "a_max": 375},
            {"a_min": 0, "a_max": 500},
            {"a_min": 450, "a_max": 750},
        ]
        new_keys = ["image1", "image2", "image3"]
        
        windowing_transforms = [
            Lambdad(keys, ScaleIntensityRanged, {"a_min": w["a_min"], "a_max": w["a_max"], "b_min":0.0, "b_max":1.0, "clip":True}) for w,keys in zip(windows, new_keys)
        ]

        remaining_transforms = [
                ConcatItemsd(keys=new_keys, name="image"),
                EnsureTyped(keys=("image", "label"), device=context.device),
                RandCropByPosNegLabeld(
                    keys=("image", "label"),
                    label_key="label",
                    spatial_size=(128, 128, 128),
                    pos=0.6,
                    neg=0.4,
                    num_samples=4,
                    image_key="image",
                    image_threshold=0,
                    allow_smaller=True,
                ),
                RandShiftIntensityd(keys="image", offsets=0.05, prob=0.5),
                RandAffined(
                    keys=['image', 'label'],
                    mode=('bilinear', 'nearest'),
                    padding_mode='zeros',
                    prob=0.8,
                    spatial_size=(128, 128, 128),
                    rotate_range=(0, 0, np.pi/18),
                    scale_range=(0.5, 0.5, 0.5)),
                SelectItemsd(keys=("image", "label")),
                ]
        return transforms + windowing_transforms + remaining_transforms

But i got this error

Lambdad(keys, ScaleIntensityRanged, {"a_min": w["a_min"], "a_max": w["a_max"], "b_min":0.0, "b_max":1.0, "clip":True}) for w,keys in zip(windows, new_keys)
File "/data/gpfs/projects/punim0406/Projects/skull-strip/monai-label-env/lib64/python3.9/site-packages/monai/transforms/utility/dictionary.py", line 1110, in init
self.inv_func = ensure_tuple_rep(inv_func, len(self.keys))
File "/data/gpfs/projects/punim0406/Projects/skull-strip/monai-label-env/lib64/python3.9/site-packages/monai/utils/misc.py", line 159, in ensure_tuple_rep
raise ValueError(f"Sequence must have length {dim}, got {len(tup)}.")
ValueError: Sequence must have length 1, got 5.

What is wrong. How does lambdad transform take original keys "key" and get 3 new keys out of it by applying the ScaleIntensityRanged transform? Can you help me out through it.
I have used in-channel=3 in the architecture. I am using monai label.

[KumoLiu]

You can do something like this:

from monai.transforms import Compose, LoadImaged, Spacingd, Lambdad, ScaleIntensityRange

transforms = [
    LoadImaged(keys='image', ensure_channel_first=True),
    Spacingd(keys='image', pixdim=(1.5, 1.5, 2.0), mode='bilinear')
]

windows = [
    {"a_min": -175, "a_max": 275},
    {"a_min": -100, "a_max": 200},
    {"a_min": -200, "a_max": 300},
    {"a_min": -150, "a_max": 250},
]

new_keys = ["image1", "image2", "image3", "image4"]
keys="image"
trans = []
for w in windows:
    trans.append(ScaleIntensityRange(a_min=w["a_min"], a_max=w["a_max"], b_min=0.0, b_max=1.0, clip=True))
windowing_transforms = [
    Lambdad([keys,]*4, trans, overwrite=new_keys)
]

final_trans = Compose(transforms + windowing_transforms)

Thanks.