Skip to content

fix(runtime): upload SenseVoice weights to selected backend - #3555

Merged
LauraGPT merged 1 commit into
mainfrom
codex/fix-vulkan-weight-buffer-3479
Aug 29, 2026
Merged

fix(runtime): upload SenseVoice weights to selected backend#3555
LauraGPT merged 1 commit into
mainfrom
codex/fix-vulkan-weight-buffer-3479

Conversation

@LauraGPT

Copy link
Copy Markdown
Collaborator

Summary

  • load SenseVoice GGUF metadata without allocating ordinary host tensor data
  • duplicate all model tensors into a context backed by the selected CPU, CUDA, or Vulkan buffer type
  • upload each weight from its exact GGUF offset before graph construction
  • read query embeddings through the backend API instead of dereferencing device-backed tensor data

Root cause

The official runtime-llamacpp-v0.2.3 Windows Vulkan executable matches the reporter's SHA-256. A symbol build from the same source maps the reported +0x2dfde3 access violation to ggml_vk_tensor_subbuffer, where src1->buffer is null before any Vulkan submission.

A graph audit found 2,386 source tensors with null backend buffers. SenseVoice loaded GGUF weights with no_alloc=false into an ordinary host context, while graph intermediates were allocated from the selected Vulkan buffer type. CPU accepted those host pointers; Vulkan requires a registered host allocation or a non-null backend buffer.

This change follows llama.cpp's model-loading pattern: metadata-only GGUF context, backend tensor context, weights buffer allocation, and explicit tensor upload.

Validation

  • python -m pytest runtime/llama.cpp/tests -q: 23 passed
  • Release CPU build against pinned llama.cpp 803b7fca
  • Release Vulkan build against pinned llama.cpp 803b7fca
  • CPU Q8 real inference: success, compute status=0
  • CPU F16 real inference: success, compute status=0
  • Vulkan Q8 real inference through llvmpipe: success, compute status=0
  • Vulkan F16 real inference through llvmpipe: success, compute status=0
  • all four runs produced the same transcript

Relates to #3479. This fixes the symbolized host-side null-buffer path, but the issue should remain open until the official Windows Vulkan artifact passes on the reporter's Radeon 780M and RX 9070 XT systems.

Signed-off-by: LauraGPT <18321252+LauraGPT@users.noreply.github.com>
@LauraGPT
LauraGPT merged commit f371370 into main Aug 29, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant