Real-time task with multiple threads.

[e950280]

I wanted it to work in real-time, so I used multiple threads to invoke the model. However, I found that the models couldn't be shared together (I'm not sure why, but it seems reasonable). So I tried two methods:

Method 1: Each thread reads a separate model.

Method 2: During the initialization of the class, the model is read, and then it is deep copied using deepcopy for further usage.

I observed that:
In Method 1, each thread starts reading the model only when the program ends.
In Method 2, the model copy works fine, but the "transcribe" process still begins only when the program ends.

I am curious about the reason behind this behavior. Thank you very much.

Below is the concept of Method 2:
I call this Class in the main program and perform some actions. However, when the main program ends, the last line "result = model.transcribe(file_name, verbose=True, temperature=0.0, language="zh", fp16=True)" starts executing.

def __init__(self):
        self.recording = False
        self.model = whisper.load_model("medium")

def _silent_processing(self,frames,file_name):
        self._whisper(file_name)

def _record(self,need_silence_detection,path):
        while self.recording:
                thread = threading.Thread(target=self._silent_processing,args=(frames,file_name,))
                thread.start()
                threads.append(thread)
        if(os.listdir(path)):
                for thread in threads:
                        thread.join()

def _whisper(self,file_name):
        model = copy.deepcopy(self.model)
        result = model.transcribe(file_name, verbose = True ,temperature=0.0 ,language="zh",fp16=True)

[glangford]

fyi

• Is all API thread-safe? (e.g. create two threads to invoke model.transcribe for two files) #296
• speed up whisper? #716