Whisper crashes in combination with multiprocessing

[molitoris]

My goal was to run whisper in a separate multiprocessing.Process. The Process dies silently (without raising an error) when it calls self.model.transcribe(audio, language='english').

The call audio = F.pad(audio, (0, padding)) in whisper.audio.py is the trigger

According to pytorch it is recommended to replace multiprocessing with torch.multiprocessing. This did not change the behavior.

How can I run whisper in a multiprocessing.Process?

System

• Kernel 6.2.0-37-generic
• Ubuntu 22.04.1
• GPU RTX 4080
• Python 3.10.12

Minimal example

import torch.multiprocessing as t_mp
import soundfile as sf
import whisper
import time


class MyModel(t_mp.Process):

    def __init__(self, queue: t_mp.Queue):
        super().__init__()
        self.model = whisper.load_model('tiny')
        self.queue = queue
    
    def run(self):
        while True:
            if not self.queue.empty():
                audio = self.queue.get()
                # The process dies on this line
                self.model.transcribe(audio, language='english')
                print(audio)


if __name__ == '__main__':
    queue = t_mp.Queue()
    model = MyModel(queue=queue)
    model.start()

    audio, _ = sf.read('recordings/sample.wav', dtype='int16')
    queue.put(audio)

    while True:
        time.sleep(1)

[qnlbnsl]

I have the same exact issue. I started using multiprocessing to run whisper and it never actually does the transcription.

[qnlbnsl]

I actually pin pointed it to the STFT under the log_mel_spectrogram function.
In my case i was not padding the audio so that block got skipped for me.
this line is where the process dies: stft = torch.stft(audio, N_FFT, HOP_LENGTH, window=window, return_complex=True)
under audio.py

[qnlbnsl]

@molitoris I think i figured out the problem. Pytorch is unable to support CUDA absed processes under multiprocessing. I had to set

import torch.multiprocessing as mp
mp.set_start_method("spawn", force=True)

[qnlbnsl]

https://pytorch.org/docs/stable/notes/multiprocessing.html#cuda-in-multiprocessing