Skip to content

Moondream example not working since 0.2.0 #215

@killzoner

Description

@killzoner

I think I tested it with v0.1.11 and it was ok at this time

Current error when running

cargo run -F vision -F vlm --example vlm -- --source ./assets/bus.jpg moondream2 --task "Vqa: What is in the image?" --visual-encoder-dtype int4

Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.16s
Running `target/debug/examples/vlm --source ./assets/bus.jpg moondream2 --task 'Vqa: What is in the image?' --visual-encoder-dtype int4`
      DryRun moondream2/0.5b-vision-encoder-int4(154.60 MiB) on CPU:0 for 3 iterations in 6s                                                                                                      
      DryRun moondream2/0.5b-vision-projection-int4 |███████             | 1/3 (285.9266/s)          2026-01-24 21:39:54.355617537 [E:onnxruntime:, sequential_executor.cc:572 ExecuteKernel] Non-zero status code returned while running MatMulNBits node. Name:'/MatMulNBits' Status Message: Input 'quantized_weight' is expected to have shape {1024,64,64}, got {8192,8,64}
Error: Non-zero status code returned while running MatMulNBits node. Name:'/MatMulNBits' Status Message: Input 'quantized_weight' is expected to have shape {1024,64,64}, got {8192,8,64}

Thanks for this great work!

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions