How to add vocabulary to the transcribe function?

[Arasun01]

I am calling the transcribe function for some medical transcripts. This model does not perform correctly for all diseases and medical terminologies. I have a list of diseases and medical terms. How do we add it as a list while calling the transcribe function? When I pass the "initial_prompt" it does not work as expected. Please advise

[phineas-pta]

can u provide a reproducible example (audio & command)

[Arasun01]

I am calling the transcribe function in the following way. I cannot share the audio file here due to the PII information. voccab_list variable should have the list of medications and diseases. It does not work as expected

import whisper
#create a model
modelwhisper = whisper.load_model("large-v2")
audio_file = <>
voccab_list = <>
result1 = modelwhisper.transcribe(audio_file,
task='transcribe',
temperature=(0.0, 0.2, 0.4, 0.8, 1.0),
best_of=5,
beam_size=3,
suppress_tokens="-1",
condition_on_previous_text=True,
fp16=True,
compression_ratio_threshold=2.4,
logprob_threshold=-1.,
no_speech_threshold=0.3,
word_timestamps=False,
initial_prompt=voccab_list)

[nshmyrev]

You can check https://github.com/BriansIDP/WhisperBiasing

[Arasun01]

Great, Thanks

[dgoryeo]

@Arasun01 , did WhisperBiasing solve for your question? I have a similar situation and was wondering about possible approaches.

[zachx121]

@Arasun01 hi, I have similar situation too, did WhisperBiasing work out?

[ghost]

I moved to a faster whisper model due to scalability issues. I did not try the WhispherBiasing with Faster Whisper model (https://github.com/guillaumekln/faster-whisper)