Releasing data suited to interactive segmentation

[daniellepace]

Is there any thinking within the MONAI community on how to best release annotated data that is suited to interactive segmentation?

We have a dataset of 60 segmented MR scans for patients with congenital heart defects. Because of the extreme anatomical variability in the disease (structures can be missing, duplicated, abnormally connected, or change in size/shape/location), state-of-the-art deep learning strategies currently fail. I think it's a really nice dataset to push the state of the art for automatic segmentation, or (perhaps more feasible) for interactive segmentation.

But, it's hard to figure out how to release it. If we do a standard challenge with best practices to hide the ground truth, we can't support projects for interactive segmentation. If we just release all of the data, we have to trust that researchers will use it appropriately, and it's hard to compare results. We're leaning towards just putting it all out there, but would love to hear any thoughts.

[wyli]

cc @aylward @IntegratorBrad @pdogra89 @csudre @ericspod @rijobro

[aylward]

Hi Danielle, @daniellepace

It is great to hear from you! Hope you're doing well, and your data sounds amazing!

I think creating a challenge involving interactive segmentation would be an outstanding contribution to the field; however, as you noted, there are logistical/metric issues with making it happen. How about contacting the leaders of grand-challenge.org (e.g., Bram Van Ginneken) and asking them for ideas? They may have encountered this in the past. Also, they are now working with AWS/Amazon, so they may have a data hosting solution for you as well.

If you would like, I could make an introduction.

Stephen

[daniellepace]

Thanks Stephen! I would love an introduction to the leaders of grand-challenge.org, if you have the time. This is a side project for us so I'm not sure if we can take on creating an entire infrastructure for interactive segmentation challenges, but if they have resources for data hosting (which we need to figure out as well) or past experiences, I'd love to hear about it. Best, Danielle

…

On Tue, Jan 10, 2023 at 7:39 AM Stephen R. Aylward ***@***.***> wrote: Hi Danielle, @daniellepace <https://github.com/daniellepace> It is great to hear from you! Hope you're doing well, and your data sounds amazing! I think creating a challenge involving interactive segmentation would be an outstanding contribution to the field; however, as you noted, there are logistical/metric issues with making it happen. How about contacting the leaders of grand-challenge.org (e.g., Bram Van Ginneken) and asking them for ideas? They may have encountered this in the past. Also, they are now working with AWS/Amazon, so they may have a data hosting solution for you as well. If you would like, I could make an introduction. Stephen — Reply to this email directly, view it on GitHub <#5815 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACTAYMDOG6COGNSDN76IN3WRVKABANCNFSM6AAAAAATSIUIXE> . You are receiving this because you were mentioned.Message ID: ***@***.***>