Interleaving transcription results for multitalker parakeet with parallel_speaker_strategy=True

[namgyaaal]

I'm testing the new "nvidia/multitalker-parakeet-streaming-0.6b-v1" on example audio files. Is there a built-in way for the utterances between different speakers to be interleaved from the seglst? With an audio file of two people speaking back-and-forth it'll have two entries in the output .json, while I would like there to be entries in order of utterances split by speaker change.

Current solution for me right now is to modify perform_parallel_streaming_stt_spk in multispk_transcribe_utils.py to return asr_state.seglsts so that I can keep track of additions per streaming buffer iteration. This is used to construct what I need at the end of transcription, but I'm not sure if this ideal.

Thanks for any help.

[tango4j]

Hi.

If you want to see frequent speaker change in the transcription, check out the following line:

NeMo/examples/asr/asr_cache_aware_streaming/speech_to_text_multitalker_streaming_infer.py

Line 97 in 66ffb38

sent_break_sec: float = 30.0

Setting a very small value for sent_break_sec=0.1 would break the sentence very often. Then you will see more interweaved transcriptions.

However - often times there is no "speaker change" because real life conversations have lots of overlapped speech. That's because the multitalker ASR model does not use the concept of "speaker change".