Separating players into teams

[Shadercloud]

I'm using detectron2 to train a model that recongizes sports players and features on the playing field (it works quite well).

Do you think it would be able to handle separating players into teams? So instead of predicting all players are "Player" class. Have it separate into "Player Team A" and "Player Team B"? (jersey colors would be varying between images, not always the same colors).

But basically if a human looked at the image they could says "These 3 objects are players on the same team" and "These other 4 objects are players on a different team".

Would detectron2 be able to handle this type of classification? I want to know if it's feasible before re-do all my annotation data to separate the players into different categories...

Currently I'm using color comparison to compare the prediction masks to try and determine which are on the same team, but results are not great, fairly slow and lots of spagetti code; would be much tidier to just handle this within the predictions.