how to unload or delete model instance

[guqun]

I created an interface for audio to text transcription requests. I load a model for each request, and after the request is complete I want to destroy the model, how should I handle it? I have tried many methods but failed to destroy the model.

[jongwook]

To fully release the model from memory, you'll need to del all references to the model, followed by torch.cuda.empty_cache() and potentially gc.collect() as well. But generally, it's not a very good idea to load the model for each request because it takes long to load the model from the disk and to the memory just to handle one request.

[guqun]

How should I use whisper in the request interface? I found that when requests are frequent, whisper will only accept one audio file and abandon the others. So my solution is to create several models for translation, but when there are too many models, I want to unload some. I don't know what the correct use of whisper should be. Please teach me, thank you