How whisper break text on phrases?

[a-ruban]

I'm trying to perform speaker diarizaion and match the results with whisper produced phrases.

As @Majdoddin suggested - I tried to insert 5 sec. silence part to audio to make Whisper break the phrase on such delimeters, but it doesn't work: whisper sometimes put both parts of delimetered phrase in one text segment.

So my questions is: how phrase breaking works in general?
Doe's anybody has suggestions how to make Whisper break the phrase in needed place?