Spanish to English translation sometimes not translate a few sentences

[junchen6072]

Hi, did anyone encounter the issue that sometimes translation isn't always working? That a few sentences are not translated.
Is this expected or is there anything I can tune? In my case, I'm translating from Spanish to English.

Sometime if I apply vad filter with a higher threshold, it could work, but also not always. Does anyone know any critical param to tune? This is my parameters:

model.transcribe(audio, word_timestamps=True, beam_size=1, vad_filter=True, vad_parameters=vad_parameters, temperature=temperature, log_prob_threshold=-1.1, no_speech_threshold=0.85, compression_ratio_threshold=2.5, condition_on_previous_text=True, task='translate')

vad_parameters = {'min_speech_duration_ms': 250, 'speech_pad_ms': 400, 'threshold': 0.3}

[glangford]

This is the openai/whisper repository, which does not have a vad_parameters. Whatever other version whisper you are using (perhaps faster-whisper) you should post your question there.

ps. beam_size=5 is the default in this repository of whisper

[junchen6072]

oh sorry, I was using faster-whisper, but it's based on openai/whisper

[glangford]

Yes it is based on it, but this code does not have vad and we don't know what changes have been made, so we can only guess here.

[junchen6072]

I see, I somehow have the impression that openai/whisper also supports VAD.
But VAD essentially is to filter out some low-confidence/noisy audio segments. With larger vad threshold, it'll be more strict so a bit more data could be filtered out.

[glangford]

You could look for the missing sentences using openai-whisper with the medium or large model, patience=2 beam_size=5. I use this for the Portuguese language with consistently very good results.

[YJYJLee]

Hi @junchen6072 , I am also experiencing the same issue. Have you resolved it?

[junchen6072]

not resolved with whisper