How to translate using Python?

[peterstavrou]

Can anyone advise how to translate a Japanese video into English for example?

I have tried:
option = whisper.DecodingOptions(language='en', task='translate', fp16=False)
and
option = whisper.DecodingOptions(language='jp', task='translate', fp16=False)
with
model = whisper.load_model('large-v2')

But both times Whisper displays the text in Japanese rather than English.

[future-leader1]

[ryanheise]

@future-leader1 's answer is factually incorrect while also sounding confident, which makes me think that the answer might have been generated by a LLM.

'jp' is not a language code, 'ja' is the correct code. But more importantly, Whisper can perform translation from other languages into English, which is exactly what @peterstavrou was asking about. That is explained in the opening sentences of the README page:

Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification.

[ryanheise]

@peterstavrou

Try using 'ja' instead of 'jp'.

[peterstavrou]

@ryanheise thanks for pointing that out. Unfortunately, I still can't get it to work and I think it's because my original code isn't passing in the options.

model = whisper.load_model('large-v2')
options = whisper.DecodingOptions(language='ja', task='translate', fp16=False)
result = model.transcribe('video.mkv')

Correct me if I'm wrong but I think I need to change the result to something like:
result = whisper.decode(model, model.transcribe('video.mkv'), options)

but with that I get the error:

  File "c:\Z\Programming\Python\OpenAI_Whisper\Video_Translate.py", line 16, in <module>
    result = whisper.decode(model, model.transcribe('video.mkv'), options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Z\Programming\Python\OpenAI_Whisper\venv\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Z\Programming\Python\OpenAI_Whisper\venv\Lib\site-packages\whisper\decoding.py", line 805, in decode
    if single := mel.ndim == 2:
                 ^^^^^^^^
AttributeError: 'dict' object has no attribute 'ndim'

[ryanheise]

You can pass the options in like this:

results = model.transcribe("video.mkv", language="ja", task="translate")

[peterstavrou]

@ryanheise thank you!