loss & mAP

[ZhenYiRan]

Hi guys, I want to train a model to detect cats, dogs and monkeys.
I used 2 models: faster-rcnn and yolo-v3.
The problem is the first model's loss is down and mAP is 0.57, while another model is hard to train.
The loss is very high and the mAP is close to 0.12 (althought I just trained 20 epoches)
I tried to change lr and use grad_clip. But it did not work well.
Can you help me to figure out what's wrong with my model?
Thank you very much!!!
(Here is my loss and mAP)

[ZhenYiRan]

Here is my config file