Multi task learning with shared backbone and different heads

[dimimal]

Hi,

I am using mmdetection library as part of my research. Im working on 2 different, but very similar tasks. I am wondering if its possible to use an instance segmentation model with 2 different heads (mask+object) and apply multi task learning on this. I guess what I have to do is to create a wrapper head class which takes the features from the Neck and add the individual heads there.

Am I write?
Can someone give me a direction?

Thanks

[SourabhRanade]

@dimimal , I am also trying to do multi task learning. I was curious if you managed to do so with mmdetection and if so could you share a couple of resources ?