-
Notifications
You must be signed in to change notification settings - Fork 40
Open
Labels
bugSomething isn't workingSomething isn't working
Description
I think I tested it with v0.1.11 and it was ok at this time
Current error when running
cargo run -F vision -F vlm --example vlm -- --source ./assets/bus.jpg moondream2 --task "Vqa: What is in the image?" --visual-encoder-dtype int4
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.16s
Running `target/debug/examples/vlm --source ./assets/bus.jpg moondream2 --task 'Vqa: What is in the image?' --visual-encoder-dtype int4`
DryRun moondream2/0.5b-vision-encoder-int4(154.60 MiB) on CPU:0 for 3 iterations in 6s
DryRun moondream2/0.5b-vision-projection-int4 |███████ | 1/3 (285.9266/s) 2026-01-24 21:39:54.355617537 [E:onnxruntime:, sequential_executor.cc:572 ExecuteKernel] Non-zero status code returned while running MatMulNBits node. Name:'/MatMulNBits' Status Message: Input 'quantized_weight' is expected to have shape {1024,64,64}, got {8192,8,64}
Error: Non-zero status code returned while running MatMulNBits node. Name:'/MatMulNBits' Status Message: Input 'quantized_weight' is expected to have shape {1024,64,64}, got {8192,8,64}Thanks for this great work!
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working