RTX 3090 vs RTX 3060 Tested whisper performance and disappointed

[FurkanGozukara]

Looks like Whisper is not optimized at all

in Stable Diffusion RTX 3090 obtains 3x performance of RTX 3060

But in Whisper?

RTX 3090 vs RTX 3060 Ultimate Showdown for Stable Diffusion, ML, AI & Video Rendering Performance

[FurkanGozukara]

@jongwook any ideas why RTX 3090 can't overwhelm RTX 3060?

[jongwook]

The implementation is likely not fully utilizing the capability of RTX 3090, because it's usually running inference on a single token at a time, and it's not doing batch inference either. The decoding logic (token suppression, logprob calculation, beam search, etc.) is also not helping because it's composed of small and sequential operations on the GPU that are not very parallelizable.

[FurkanGozukara]

Thanks a lot for the answer. Makes sense. Any plan for model V3 or whisper v2? For some reason model v1 works better for my speech. It is English