Running out of RAM --though model runs on GPU

[dgoryeo]

Hi, while trying to transcribe several files I run out of RAM. It looks like Whiper does not release RAM after each call. I have tried using del and gc but no effect. What am I doing wrong?

Below is the code. The environent is colab pro and the model runs on T4 GPU.

#defining transcription call function
def run_Whisper(input_audio_file):
    model = whisper.load_model(model_size)
    result = model.transcribe(input_audio_file, **options)

# Define a generator function to generate all audio files in a directory
def audio_files_generator(directory_path):
    for file_name in os.listdir(directory_path):
        if file_name.endswith(".mp3") or file_name.endswith(".wav"):
            yield os.path.join(directory_path, file_name)


audio_files_list = list(audio_files_generator(directory_path))

# Call the process_audio_file function for each audio file in the list
for audio_file in audio_files_list:
    run_whisper(audio_file)

Thanks!

[glangford]

It looks like you are reloading the model multiple times.

In this code

for audio_file in audio_files_list:
    run_whisper(audio_file)

run_whisper will load the model every time it is called. Instead, you want to load the model once, then iterate through the files reusing that model. eg.

model = whisper.load_model(model_size)
for audio_file in audio_files_list:
   result = model.transcribe(audio_file, **options)

[dgoryeo]

Thanks @glangford. You're right it was by design to load the model inside the run_whisper function. I expected Whisper to release memory after each function call, but that doesn't seem to happen here.

The motivation behind that design was to compare the outputs for each run while the model has started "afresh" for each. Is there a way for me to "reset" the model for each run/file without reloading the model?

Kind regards

[glangford]

Is there a way for me to "reset" the model for each run/file without reloading the model?

Loading the model once and using it for multiple files is the way that whisper does it in command line processing.

whisper/whisper/transcribe.py

Line 455 in e8622f9

for audio_path in args.pop("audio"):

There isn't any reset mechanism I am aware of, but note that you can get different results even when transcribing the same file. So if the idea is to get the model to a "known good state" to get predictable outputs, that isn't guaranteed. See

• Different transcribe results using same model to transcribe the same audio #81

On Colab, the large model runs fine on the Tesla T4 for example.