Using PatchInferer with masks

[mdrio]

Hi, I need to do inference on whole slide images using some binary mask at a given level of magnification, basically using it for filtering the slide. How can I achieve it using PatchInferer?
In the case the mask is the foreground, if I understand well MaskedPatchWSIDataset could be used. Anyway I am not sure how to combine it with the inferer.

Thanks in advance

[KumoLiu]

Hi @mdrio, thanks for your interest here.
You can find some examples in this tutorial.
Hope it helps, thanks!

[mdrio]

The tutorial shows how to filter patches before inference. In my use case I have a list of points that represents a polygon on which I want to run a predictor. So filtering patches using filter_fn does not seem suitable.

I suppose MonaiLabel does something similiar with the wsi_infer interface, anyway I wonder how to implement that functionality on a lower level than MonaiLabel.

[mdrio]

The regions are annotated by pathologists and are subregions of the foreground. Basically it is something like this: https://slideflow.dev/slide_processing/#regions-of-interest

[KumoLiu]

Hi @mdrio, we only have BoxToMask which can help convert box to int16 mask image, but didn't support polygon. You could consider using cv2.fillPoly to help convert the points to mask.
A simple demo here.

import numpy as np
import cv2

spatial_size = (10, 10)
im = np.zeros(spatial_size, dtype="uint8")
cor_xy = np.array([(1, 4),(3, 4), [2, 5], [4, 5], [3, 6]], 'int32')

cv2.polylines(im, np.int32([cor_xy]), 1, 1)
cv2.fillPoly(im, np.int32([cor_xy]), 1)

mask_array = im

Then you could use CropForeground to help get the image based on the mask.
Thanks!

[mdrio]

Sorry, I'm not sure how to use the obtained mask with CropForeground in order to extract patches from a whole slide image (so I don't want the full image on memory).
I assume I have to write a function that uses the mask array and pass it to CropForeground as select_fn parameter. But how can I extract patches then?

thanks

[KumoLiu]

I think the fastest workaround so far might be to change these two lines to use the mask you generated we mentioned above.

MONAI/monai/data/wsi_datasets.py

Lines 397 to 399 in 782c1e6

	# create the foreground tissue mask and get all indices for non-zero pixels
	mask = np.squeeze(convert_to_dst_type(ForegroundMask(hsv_threshold={"S": "otsu"})(wsi), dst=wsi)[0])
	mask_locations = np.vstack(mask.nonzero()).T

Perhaps we should consider enhancing how masks are generated or adding more flexibility here such as accepting the user-provided mask. cc @drbeh

[mdrio]

Ok I'll try to implement a modified version of MaskedPatchWSIDataset. Thanks for your suggestions!