Transcription fails with AttributeError when attempting to transcribe a torch array

[MrEnigmamgine]

Hi. To save on memory I'm trying to transcribe only short segments at a time. However, whenever I feed a torch array into the transcribe function the process errors out with the following error:

AttributeError                            Traceback (most recent call last)
Cell In[11], line 1
----> 1 base.transcribe(audio=wav)

File /opt/conda/lib/python3.10/site-packages/whisper/transcribe.py:181, in transcribe(model, audio, verbose, temperature, compression_ratio_threshold, logprob_threshold, no_speech_threshold, condition_on_previous_text, **decode_options)
    178 segment_duration = segment.shape[-1] * HOP_LENGTH / SAMPLE_RATE
    180 decode_options["prompt"] = all_tokens[prompt_reset_since:]
--> 181 result: DecodingResult = decode_with_fallback(segment)
    182 tokens = torch.tensor(result.tokens)
    184 if no_speech_threshold is not None:
    185     # no voice activity check

File /opt/conda/lib/python3.10/site-packages/whisper/transcribe.py:120, in transcribe.<locals>.decode_with_fallback(segment)
    117 decode_result = model.decode(segment, options)
    119 needs_fallback = False
--> 120 if compression_ratio_threshold is not None and decode_result.compression_ratio > compression_ratio_threshold:
    121     needs_fallback = True  # too repetitive
    122 if logprob_threshold is not None and decode_result.avg_logprob < logprob_threshold:

AttributeError: 'list' object has no attribute 'compression_ratio'

Here is the code I'm using to reproduce the error:

import whisper
import torchaudio as ta

print(whisper.__version__) # 20230124
print(ta.__version__) # 0.13.1+cu116

model_dir = '../models/whisper'
base = whisper.load_model('base.en', download_root= model_dir)

wav, sr = ta.load('../data/db short intro.ogg')
print(type(wav)) # torch.Tensor

base.transcribe(audio=wav)

[jongwook]

Hi, I think the model is treating each channel separately and running a batch inference, hence returning a list instead of a DecodingResult instance.

I'd recommend using whisper.audio.load_audio() function, which takes care of converting the sampling rate to 16,000Hz, as expected by the model.