Interest in 16. Traffic Intersection Monitoring System with OpenVINO

[jordanttay1]

Hi, my name is Jordan. I'm a first year masters student at Georgia Tech studying CS with a concentration in machine learning. I have an internship experience in which I used ML at a fintech startup, and developed a UI from scratch using QT.
Additionally, I've created my own research paper, where I applied traditional ML and DL methods to predict pricing in illiquid financial markets.

I was hit by a car which ran a red light a few years ago, so this idea really caught my attention.

I have a few questions about the structure and scope of this project.

• To what extent are the computer vision methods known and how restricted is compute? I've played around with TensorFlow, and LLMs constantly seem to be getting better at handling images. In a world with infinite resources, it seems that LLMs may be best, but that's generally not realistic.
  -What types of data do we get and need to be able to detect? I.e. will we be given an indicator for a red light or will the light be in frame? Will we know if a car is speeding?
  -Are there any plans to handle adverse situations such as heavy rain?

Thanks in advance for your answers.
@adrianboguszewski @mlukasze Can you please help me find the mentors? It should be Dan Qiu and Yiwei Lee Thank you!

[adrianboguszewski]

@dan20210809

[jordanttay1]

Thanks!

[dan20210809]

Hi, Jordan, thank you for your questions, please see my comments:
-To what extent are the computer vision methods known and how restricted is compute? I've played around with TensorFlow, and LLMs constantly seem to be getting better at handling images. In a world with infinite resources, it seems that LLMs may be best, but that's generally not realistic.
-------The project is so far built on traditional object detection algo not LLMs, one of the basic reasons is the compute ability. We are going to use OpenVINO to deploy the model on Intel platform (Ultra5 125H would be the demo platform). Although LLM is not in the plan, but we can try to extend the project to LLM if we have enough time, because OpenVINO can accept LLM models.

-What types of data do we get and need to be able to detect? I.e. will we be given an indicator for a red light or will the light be in frame? Will we know if a car is speeding?
-------We will have indicators for red light, car, people etc, but our focus is not on how to write the best algo to process the detection results, but is on how to optimize the model and increase the pipeline latency. This project is actually the vision part of the V2X(Vehicle-to-Everything) solution, in which a lot of information will be used to determine the traffic violation, we use the project to showcase the deployment skills and if possible we want to optimize the algo for example by using LLMs.

-Are there any plans to handle adverse situations such as heavy rain?
-------For the adverse situations, the AI detection result is not enough, we have to use other data such as radar data with car distance and direction to increase the accuracy, which is not included in this project. So far we don't have plan for it.

[jordanttay1]

That's all very interesting. Thank you Dan for such a thorough response!
I'm working on a proposal for it now. Hopefully I'll get to work more on this idea soon.