Hello and apologies if this is the wrong place to file an improvement suggestion.
I am running HomeAssistant in Proxmox with an AI assistant (via Ollama on a separate VM) and often use the voice assistant (using Whisper and Piper). This works pretty well -- with the small but annoying issue that it often takes rather long before the AI response can be heard.
It appears the problem is that the voice generation only starts after the text response is complete. Even with a very fast LLM this can take a relatively long time, during which the user just has to wait.
The response time could be much improved, if there was an option to start generating the audio as soon as a certain number of words (or, e.g. the first sentence, the first paragraph, etc.) is finished.
I believe that this should be an option - in some cases, e.g. when the LLM runs locally on the CPU - it probably makes sense to wait until the response is complete. In case of a remote AI, and/or a very fast text generation, there is no problem to parallelize these tasks, thus improving the user experience.
It would be great if this could be considered as an improvement to this feature - even though this comes already after the "Year of Voice".
Hello and apologies if this is the wrong place to file an improvement suggestion.
I am running HomeAssistant in Proxmox with an AI assistant (via Ollama on a separate VM) and often use the voice assistant (using Whisper and Piper). This works pretty well -- with the small but annoying issue that it often takes rather long before the AI response can be heard.
It appears the problem is that the voice generation only starts after the text response is complete. Even with a very fast LLM this can take a relatively long time, during which the user just has to wait.
The response time could be much improved, if there was an option to start generating the audio as soon as a certain number of words (or, e.g. the first sentence, the first paragraph, etc.) is finished.
I believe that this should be an option - in some cases, e.g. when the LLM runs locally on the CPU - it probably makes sense to wait until the response is complete. In case of a remote AI, and/or a very fast text generation, there is no problem to parallelize these tasks, thus improving the user experience.
It would be great if this could be considered as an improvement to this feature - even though this comes already after the "Year of Voice".