Landmark detection

[dyollb]

Hi

I would like to train a network to predict landmark positions in head MRI (four reference positions on skin to place 10-10 system electrodes). I read the discussion and issues, and found following relevant discussion: #1169

My idea is to

• convert the 4 landmark positions to 3d images with 4 channels, e.g. a sigmoid at the landmark positions
• use Attention UNet with a MEA loss or maybe with a Euclidean distance of between landmark positions
• for convenience, maybe implement transforms that convert positions to image/and extract positions from images

Or would the bounding box detection feature (@Can-Zhao) provide this kind of functionality?

Thanks for any ideas or pointers to relevant examples...
Bryn

[wyli]

I think @diazandr3s is working on a similar landmark regression task: Project-MONAI/MONAILabel#855

[dyollb]

thanks @wyli for pointing me to this. looks useful

[diazandr3s]

Yes, this is something we're discussing here: Project-MONAI/tutorials#100 (comment)
The current vertebra multistage model uses only segmentation tasks, but we plan to have a regression model for the second stage.

[dyollb]

@diazandr3s according to that discussion, it sounds like monai.UNet + regression is not working very well, and that you are using segmentation. But if I look at the code it I see you are using torch.nn.MSELoss and I don't see you are not converting the pred to labels (only leaky_relu). Isn't that a regression?

[diazandr3s]

Hi @dyollb,

I see you are using torch.nn.MSELoss and I don't see you are not converting the pred to labels (only leaky_relu). Isn't that a regression?

You're right, that's regression. However, the final version does not use regression. Here is the final version on the main branch: https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/trainers/localization_vertebra.py#L64

For this particular task, I've changed from regression to segmentation because I've got better results. But I guess this depends on the task and the needs. Happy to share my experience on a call.

[dyollb]

Great. Thanks for your answers! I will give it a try.
Btw. do you think the AttentionUNet will give similar results to the two stage approach of first finding the ROI, then detecting the landmark in the ROI?

[diazandr3s]

Great. Thanks for your answers! I will give it a try.

Superb! Please keep us posted.

Btw. do you think the AttentionUNet will give similar results to the two stage approach of first finding the ROI, then detecting the landmark in the ROI?

This is a good question - I haven't compared both strategies myself. I'd say the multiple stages approach could work better than the single ones - mainly because of the divide and conquer strategy :)

[dyollb]

thanks @diazandr3s! For the segmentation approach, I guess you annotated the vertebrae centers (training data) with a radius, since Dice wouldn't make much sense for a single voxel. What radius did you choose (relative to voxel size)?

[diazandr3s]

I just used the vertebra annotations on the VerSe dataset. Once the model is trained, I use the predictions obtained from the model to compute the vertebra centroids which are used in the third stage. I hope this makes sense.