Issues related to tracking

[melinasolalvarez-blip]

I am using S-LEAP to track the movement of up to 21 Drosophila flies simultaneously. I have been following the online instructions carefully. Because the quality of my videos is not ideal, I labeled 220 suggested frames, and up to that point, the model performed quite well. I created a skeleton with five nodes, and the predictions looked accurate. However, I began encountering problems during the tracking step. The fly identities were often switched, and multiple tracks were generated. To address this, I deleted all tracks except the first one for each fly and manually corrected them for the first 138 frames. When I proceeded to train the model afterward, the training process took a long time, and the subsequent inferences produced worse results than before — including errors such as missing one of the flies entirely in certain frames. I would greatly appreciate any advice you could offer on how to proceed. I have been working on this for the past few weeks, and it seems that the more I train the model, the worse the results become.
Here's a video (sorry for the resolution it had to be less than 10 MB) and the captures of configurations I used for reference.

sleap.mp4

Thank you very much for your time.

[gitttt-1234]

Hi @melinasolalvarez-blip !

Tracking quality depends heavily on how well the pose model detects every fly consistently. Identity switches and extra tracks usually happen when some flies are missed or their poses jump between frames. For improving the pose model: For Centroid model, try increasing the number of filters to 32, especially if earlier predictions were missing flies. For the Centered Instance model, set Input Scaling = 1.0 — since the model already works on cropped patches, additional scaling can actually negatively impact performance. While running tracker, try with a different similarity method: instance / oks-based would be better than centroid.

Because you're not using a Top-Down ID model, the tracking that happens after inference is purely heuristic. This means you don’t need to manually correct or label tracks — they are not used for training, and correcting them won’t improve the model. Only the keypoint labels affect training. You mentioned that training afterward produced worse results. How do the evaluation metrics look for each training run, are the metrics getting worse?

Let us know if you have any questions!

Thanks,

Divya