[feature request] Returning tracks that are alive but occluded

[sunildkumar]

I've been working on integrating your SORT/ Deep SORT trackers into an application, but I've found a confusing design decision. It looks you expect us to use the output of tracker.update(detections) to represent the tracks present on the current frame. update only returns the subset of tracks that match to the detections that were available for the given frame. This "hides" tracks that are still alive but may be temporarily occluded or spuriously not detected.

Does it make sense to provide a method that returns all alive tracks given the current tracker state?

[soumik12345]

Hi @sunildkumar, This is a good idea!
An ideal tracker is supposed to track objects despite occlusions and detection failures, so we should ideally make all the alive tracklets accessible. However, it should probably not be returned from tracker.update; it's better to have a separate property, like tracker.alive_tracklets, which should give users access to the state of alive tracklets.

[sunildkumar]

Hey there! Thank you for your quick reply. That seems like an acceptable way to expose this information.

[SkalskiP]

Hi @sunildkumar 👋🏻

Are you still interested in this feature?

[sunildkumar]

Yes I think it would be helpful to expose this for many CV applications.

[SkalskiP]

Hi @sunildkumar, glad to hear this is still relevant! Here's what I propose.

Add a tracked_objects property to our BaseTracker class. After calling update(), you'll be able to access all alive tracks (including occluded/unmatched ones) via their Kalman-predicted bounding boxes.

# BaseTracker
@property
@abstractmethod
def tracked_objects(self) -> sv.Detections:
    """All alive tracks with Kalman-predicted bounding boxes."""
    ...

This keeps update() unchanged and non-breaking. We'll implement this across both SORTTracker and ByteTrackTracker. Stay tuned!

[sunildkumar]

perfect! thank you.

[SkalskiP]

Please stick with Trackers, @sunildkumar! I know it took us some time to get this project off the ground, but we are moving fast now.