frame drop when increase number of RTSP stream

[lalitbyteiq]

I'm using YOLO v8 for tracking and counting vehicles on RTSP streams. I have around 50 streams, each running at 25 FPS. I'm employing multiprocessing, and while processing 5 streams works smoothly, increasing the number of streams leads to significant frame skipping.

I've tried the following approaches:

Separate Processes for Capturing and Processing:
One process captures frames and adds them to a queue.
Another process handles processing and prediction.
Combined Process for Capturing and Processing:
A single process handles both capturing and processing frames.
Using Kafka for Frame Handling:
Capture frames and send them to a Kafka topic for all streams.
Use a single consumer to process all frames.
However, this approach may also face frame skipping issues as the number of streams increases.
System Specifications:
RAM: 32 GB
Storage: 2 TB
Processor: i7 12th Gen
GPU: RTX 3060 (12 GB dedicated, 16 GB shared)

I'm seeking advice on how to efficiently handle a higher number of streams without skipping frames. Any suggestions or best practices would be greatly appreciated.

[crackwitz]

related: https://forum.opencv.org/t/frame-drops-when-increasing-the-number-of-rtsp-streams/18292 and ultralytics/ultralytics#14495

[SkalskiP]

Hi @lalitbyteiq 👋🏻, have you tried InferencePipeline?