🐛 Bug Description
i successfully install shimmy via cargo install shimmy --features gpu
~/.cargo/bin/shimmy list show me some already downloaded models
but i can't serve or probe any
it get always probe failed: null result from llama cpp
🔄 Steps to Reproduce
~/.cargo/bin/shimmy probe phi3-lora
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GB10, compute capability 12.1, VMM: yes
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GB10) (000f:01:00.0) - 73847 MiB free
gguf_init_from_file: failed to open GGUF file './models/phi3-mini.gguf'
llama_model_load: error loading model: llama_model_loader: failed to load model from ./models/phi3-mini.gguf
llama_model_load_from_file_impl: failed to load model
probe failed: null result from llama cpp
✅ Expected Behavior
loading successfully models
❌ Actual Behavior
probe failed: null result from llama cpp
📦 Shimmy Version
~/.cargo/bin/shimmy --version
shimmy 1.8.2
💻 Operating System
Linux (Ubuntu)
📥 Installation Method
cargo install shimmy
🌍 Environment Details
rustc --version
rustc 1.91.1 (ed61e7d7e 2025-11-07)
hardware NVIDIA GB10
🐛 Bug Description
i successfully install shimmy via
cargo install shimmy --features gpu~/.cargo/bin/shimmy listshow me some already downloaded modelsbut i can't
serveorprobeanyit get always
probe failed: null result from llama cpp🔄 Steps to Reproduce
~/.cargo/bin/shimmy probe phi3-lora
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GB10, compute capability 12.1, VMM: yes
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GB10) (000f:01:00.0) - 73847 MiB free
gguf_init_from_file: failed to open GGUF file './models/phi3-mini.gguf'
llama_model_load: error loading model: llama_model_loader: failed to load model from ./models/phi3-mini.gguf
llama_model_load_from_file_impl: failed to load model
probe failed: null result from llama cpp
✅ Expected Behavior
loading successfully models
❌ Actual Behavior
probe failed: null result from llama cpp
📦 Shimmy Version
~/.cargo/bin/shimmy --version
shimmy 1.8.2
💻 Operating System
Linux (Ubuntu)
📥 Installation Method
cargo install shimmy
🌍 Environment Details
rustc --version
rustc 1.91.1 (ed61e7d7e 2025-11-07)
hardware NVIDIA GB10