Skip to content

Conversation

@ggerganov
Copy link
Member

fix #13712

  • Set the WavTokenizer n_ubatch == n_batch since it is computing embeddings
  • Use llama_encode() instead of llama_decode() with the WavTokenizer context
  • The WavTokenizer does not need a KV cache, so we now no longer create it. This saves about ~800 MB of memory.

@ggerganov ggerganov force-pushed the gg/tts-fix-ubatch branch from b814fb7 to 108d484 Compare May 22, 2025 18:58
@ggerganov ggerganov merged commit 8a1d206 into master May 22, 2025
51 of 53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Eval bug: llama-tts fails (abort) with longer lines

2 participants