What is the best way to transcribe a conversation?

[fanyangxyz]

I have a recording of a therapy session, about 30-min conversation between a patient and her therapist. I used whisper to transcribe but the result is a long blob of text, not in the dialog format. I also tried using the prompt of "This is a conversation between ...". But it did not change the result much. Is whisper capable of transcribing conversation into dialog format? By dialog format I mean something like
"Person A: ..."
"Person B: ..."

[Tex2002ans]

I used whisper to transcribe but the result is a long blob of text, not in the dialog format. [...] Is whisper capable of transcribing conversation into dialog format?

"Person A: ..."
"Person B: ..."

No. Whisper can't currently do this.

The technical term for this is called:

• Speaker Diarization.

You can find discussion on this in other topics:

• Speaker diarization (partitioning audio based on speaker identity) #104
• Transcription and diarization (speaker identification) #264
• https://github.com/openai/whisper/discussions?discussions_q=is%3Aopen+diarization

Currently, the closest answer was given by jongwook:

• prompt vs prefix in DecodingOptions #117 (comment)

where you can "hack" initial_prompt with hyphens, in order to nudge Whisper to potentially output dashes between speakers.

[...] You can also use this for "prompt engineering", to inform the model to become more likely to output certain jargon (" So we were just talking about DALL·E") or do a crude form of speaker turn tracking (e.g. " - Hey how are you doing? - I'm doing good. How are you?", note that the token for " -" is suppressed by default and will need to be enabled manually.)

Besides that, you may need to use/program external tools—like Pyannote—to try to take:

• Original Audio + Whisper's transcripts

which will generate timings, which would help tag certain lines as Speaker A + Speaker B.

(See those previously linked topics for much more details.)

[fanyangxyz]

Thanks so much for the information. I'll look into them.

[Majdoddin]

@fanyangxyz @Tex2002ans
www.lexicaps.com adds diarization to Whispers transcription. No 3rd party packages.
Announcement: #1537
Repo: https://github.com/Majdoddin/lexicaps

[Frosti7]

Yes, it does support speaker - Speaker Diarization. https://github.com/lablab-ai/Whisper-transcription_and_diarization-speaker-identification-

…

On Tue, 4 Apr 2023, 11:54 Fan Yang, ***@***.***> wrote: Thanks so much for the information. I'll look into them. — Reply to this email directly, view it on GitHub <#1191 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJRADHDEBFJHJGVIP3L3SZDW7OLOPANCNFSM6AAAAAAWR7X7IA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Joshfindit]

Interested to see if this has changed with the release of Whisper 3

[envious]

Same