question on speaker detection and speaker differentiation

[ykarous]

Is the detection function of the speakers functional and is their differentiation the same as in the original model?

[senlyu163]

Hi, the project did not implement speaker detection and speaker differentiation.

The initial goal was to implement local inference on the orin-nano and reach real-time communication performance. Because the orin-nano's VRAM which program could use is less than 8GB, even after quantizing all models, it was impossible to implement all the modules of MiniCPM-o 2.6 on the orin-nano. The final result is shown in the video demo, with speaker detection replace by a single web button.

If your device has more than 8GB vram, you can manually implement speaker detection and other functions. We provide C++/Python interfaces for easy expansion.

[ykarous]

Okay, but are speaker detections on the gguf files you provide available?

[senlyu163]

Unfortunately, this project did not attempt to implement speaker detections on the gguf.