Intel NUC to run Whisper?

[robdejonge]

I am currently looking for a new home server, and one of the key factors is being able to run Whisper tasks as fast as possible. I am however not sure what resource I should configure more of, to have Whisper run faster. For clarity: looking at Home Assistant's Whisper integration.

Background: I am looking at a few different Intel NUCs at the moment. One uses a quad-core Celeron N5105 (Geekbench multi ~1500) with up to 32GB, the other a dual-core i3 processor (Geekbench multi ~3200) with up to 64GB worth of RAM.

Q1: Will the Celeron suffice for "Google Assistant-speed replies" or would the faster cpu result in a significantly better experience?

Q2: Also, I noticed in the readme VRAM requirements where it seems more of that makes for better results ... I'm not sure if this implies the need for a beefy GPU with lots of memory, or if system memory can somehow be used for this purpose?

Again my apologies for the offtopic use of this forum - comments would be very much appreciated!

[phineas-pta]

if u dont have gpu it's slower, especially celeron is lower end cpu, u should use tiny.en model if unable to get powerful hardware

[robdejonge]

For further explanation, "Mike H." over on the Home Assistant Discord offered the following explanation which I'm re-posting here as it came in around the same time and offers a tad bit further clarification:

"Both [RAM and CPU resources make a difference] up to a point. A few GB's of RAM and almost any Intel/AMD CPU will run Whisper well. To run the largest and most accurate models, you will want a GPU though." (link)

"The math underneath Whisper, etc. is done really quickly by GPUs but can also be done more slowly by CPUs. For small models, it doesn't matter much because the speed differences are tiny. But for larger models (more accurate), a GPU can get done in milliseconds what takes a CPU several seconds." (link)

Thanks @phineas-pta for your response!