Implement SLEAP on realtime processing

[huyle24022002]

Hi, I want to ask about how to deploy SLEAP on real time capture. My current pipeline are: Have a trained model, capture from camera, run the model on each frame of capture. The problems is the fps is just around 11fps. Beside that, I used SLEAP 1.5.1 without support of sleap.load_model() function as in real time inference example in this gg colab https://colab.research.google.com/github/talmolab/sleap/blob/main/docs/notebooks/Interactive_and_realtime_inference.ipynb.

Really appreciate with any suggestions

[alicup29]

Hey huyle, sorry for the late response!

Unfortunately, we currently don't support real-time capture yet in SLEAP 1.5+, because with SLEAP 1.5+, we recently migrated from TensorFlow to PyTorch. However, we are planning to do a refactor to support real-time inference soon. SLEAP 1.5+ can currently only take in "SLEAP objects" (i.e. Labels, Video (mp4), etc.), so you could continue to use older SLEAP versions, like the in the colab notebook. If you have any other questions about running inference SLEAP 1.5+, feel free to reach out!

Sorry again for the late response!

Thank you,
Amick L.

[HuyLeeDuck]

Hi ali,

May I know the plan to release the realtime inference support for SLEAP 1.5+? Will it release on this year (2026). Since our data are labeled on SLEAP 1.5+. So, I'm afraid about the back compatible with legacy SLEAP version.

Thank you!