[Performance/Vision] Implement multi-threaded VideoCapture to unblock main inference loop

[Viidhii19]

Problem / motivation

Currently, cv2.VideoCapture.read() is called synchronously in the main vision loop. Because camera I/O is a blocking operation, the main thread halts and waits for the hardware to return the next frame. This creates artificial latency, slowing down the overall gesture inference pipeline and dropping the effective FPS.

Proposed solution

Decouple the camera I/O from the inference loop by implementing a threaded video capture class. A daemon thread should continuously read frames from the camera into a shared variable, allowing the main gesture loop to instantly grab the freshest frame without waiting for hardware I/O.

Implementation notes

1. Create a ThreadedCamera class in a new utils/camera.py file.
2. Initialize a threading.Thread(target=self._update, args=()) running as a daemon.
3. The _update method continuously reads cap.read() into an instance variable self.frame.
4. The main loop simply calls a non-blocking read() method that returns self.frame instantly.

Would you like to work on this?

• Yes, I'd like to implement this multi-threading optimization! (GSSoC contributor)

[Viidhii19]

Hye @VyomKulshrestha !
please assign this to me!

[anshul23102]

Hi @VyomKulshrestha, important performance enhancement! Implementing multi-threaded VideoCapture would unblock the main inference loop and improve responsiveness.

Could this be assigned to me? I'm ready to implement thread-safe video capture.

Excited to help! 🎥