Skip to content

fix(runtime): decode SenseVoice F16 query embeddings - #3550

Merged
LauraGPT merged 1 commit into
mainfrom
codex/fix-sensevoice-empty-output
Aug 29, 2026
Merged

fix(runtime): decode SenseVoice F16 query embeddings#3550
LauraGPT merged 1 commit into
mainfrom
codex/fix-sensevoice-empty-output

Conversation

@LauraGPT

Copy link
Copy Markdown
Collaborator

Summary

  • decode SenseVoice query embeddings according to their GGML tensor type instead of reinterpreting F16 storage as float*
  • validate the embedding width and query-token bounds before inference
  • update the SenseVoice numerical golden to the corrected ITN output

Root cause

The published F16 GGUF stores embed.weight as F16 [560, 16], while the runtime read it as F32. Query-token copies therefore read twice the actual storage width and crossed tensor bounds. Depending on process memory contents, CTC intermittently collapsed to blank tokens and returned an empty transcript with exit code 0.

Related adopter report: huaxin0/FunASR-GGML#4

Validation

  • reproduced with the exact runtime-llamacpp-v0.2.3 Linux AVX2 asset: 22 newline-only outputs among 298 completed sequential runs
  • red/green source contract test for F16/F32 embedding decoding
  • 100/100 fixed F16 runs: zero empty outputs, one output hash, all compute statuses successful
  • corrected F16 output matches the Q8 model F32-embedding output byte-for-byte
  • python3 -m pytest -q runtime/llama.cpp/tests/*.py: 22 passed
  • run_regression.sh with the F16 model: SenseVoice passed
  • Release build of llama-funasr-sensevoice succeeded

Signed-off-by: LauraGPT <18321252+LauraGPT@users.noreply.github.com>
@LauraGPT

Copy link
Copy Markdown
Collaborator Author

Exact-head release matrix completed successfully on 178a36718cb1110255296bdf4986ff4ebf295019:

  • 9/9 build jobs succeeded
  • Linux x64, AVX2, Vulkan, ARM64
  • macOS ARM64
  • Windows x64, AVX2, Vulkan, CUDA

Evidence: https://github.com/modelscope/FunASR/actions/runs/33229633488

@LauraGPT
LauraGPT merged commit f04b761 into main Aug 29, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant