How to segment 3D images of very large resolution ?

[scarpma]

Hello,
I would like to segment CT scans of very high resolution, like 512x512x700. Since fitting everything into the gpu during training doesn't seem the best solution, even if the memory limit was not a problem, I would like to know if there are some already implemented features to face such a problem.

One idea that comes to my mind is to do a segmentation at low resolution (from like 0.5 mm/pixel to 1 mm/pixel) to first asses the main geometry. Then, another network could be evaluated through a sliding window inferer at sub-regions around the low resolution segmentation in order to have a better resolution. Are there any already implemented function to do this ?

A basic approach would be to go for full resolution from the beginning, using a sliding window inferer, but with a small ROI. The problem here is that the network would have a very limited field of view of the sorroundings... Is there a standard solution for this problem ?

Thank you very much !

[Nic-Ma]

Hi @dongyang0122 ,

Could you please help share your practice experience for this question?

Thanks in advance.

[dongyang0122]

@scarpma you can train a low resolution model first, and use the predictions of the first model together with images to train a high-resolution model. monai.transforms.CropForeground can be used here to crop foreground regions with a pre-defined margin within high-resolution images.

[scarpma]

This seems interesting! I see that monai.transforms.CropForeground selects a bounding box. Of course, this would already help a lot. One problem is that the object I'm interested in is the aorta, which is a pretty thin but long tube inside the whole chest. The resulting bounding box would still contain a lot of background (as you can see from the image).

Is there a way to use something like a sliding window that processes only non-empty windows ? The pipeline that I imagine is this one

[low res image] ==> [low res net] ==> [low res seg] ==> [slicing window inferer] ==> [high res net] ==> [high res seg]

I imagine passing a sort of mask to monai.inferers.sliding_window_inference in order to skip empty volumes, according to the [low res seg].

[scarpma]

Maybe one important question is how to select sub regions of a 3d volume of a certain ROI size, given a segmentation mask. I think that if there is no segmentation mask, the implementation is pretty easy (should be inside the source code of monai.inferers.sliding_window_inference). But with a segmentation mask it is not so immediate.

My idea is something like this (3d volumes instead of quads):