Custom Object Detection works with RetinaNet but fails with Fast R-CNN or Mask R-CNN

[gul15]

Hello everyone, I have been trying to develop a vertebra detector using detectron2. I developed a simple one using RetinaNet from detectron2 model zoo.
The input annotated data looks like this:

Also the output from RetinaNet:

But when I try to use other detection models such as Faster R-CNN R50-FPN or Instance Segmentation models it does not give sensible outputs.

I would appriciate any help! Thanks.

[zhanghang1989]

This might be related to the anchors set up. You may try to adjust the anchor settings