-
Notifications
You must be signed in to change notification settings - Fork 13.4k
llama : add support for Deepseek-R1-Qwen distill model #11310
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
cc @bartowski1182, you can now make GGUF quants :D |
Are similar changes needed to support |
@prusnak I don't have time to try, but there are already many GGUFs for that model on the HF hub. Can you try? |
I just tried |
|
I saw a similar (though reversed) issue with lmstudio, where the model sends one response and then crashes in the chat, but the server works fine 🤔 |
* llama : add support for Deepseek-R1-Qwen distill model * coding style
* llama : add support for Deepseek-R1-Qwen distill model * coding style
* llama : add support for Deepseek-R1-Qwen distill model * coding style
* llama : add support for Deepseek-R1-Qwen distill model * coding style
Support all Qwen-based model of the Deepseek-R1 distill family: