2025/04/16 Meeting Notes

[himanshunaidu]

Progress Update

1. Executorch Research

Explored ExecuTorch which is an end-to-end solution for enabling on-device inference capabilities across mobile and edge devices including wearables, embedded devices and microcontrollers.
Identified benefits as well as issues while testing the conversion/runtime and going through the documentation.
More info here: #45

2. Computer Vision ML Pipeline

Continued working further on the testing logic of the ML pipeline.
Here are the evaluation metrics for the main options that have been tried so far:

(Note: Need to redo the ROM/RUM scores for all models except BiSeNetv2 CoreML and Executorch 1024 x 512)

Model	Input Size	Source	mIoU	ROM	RUM
ESPNetv2	1024 x 512	PyTorch	0.6313	0.1195	0.1171
ESPNetv2	512 x 256	PyTorch	0.4789	0.0990	0.2800
BiSeNetv2	1024 x 512	PyTorch	0.6727	0.0776	0.3248
BiSeNetv2	512 x 256	PyTorch	0.5531	0.0468	0.5234
BiSeNetv2	1024 x 512	CoreML	0.6814	0.0712	0.3096
BiSeNetv2	512 x 256	CoreML	0.5666	0.0504	0.5120
BiSeNetv2	1024 x 512	Executorch	0.6814	0.0712	0.3117
BiSeNetv2	512 x 256	Executorch	0.5666	0.0509	0.5117

Note: The ROM/RUM scores may be incorrect due to some environment issues with skimage. Need to verify it soon

[himanshunaidu]

Computer Vision ML Pipeline

Qualitative Performance

Id	Original	Prediction
103
444
453
465