I am using an AMD GPU to run the deepseek_r1_distill_llama_70b_q40 big model, why is there no output in the conversation.

[nodepend-code]

I am using an AMD GPU to run the deepseek_r1_distill_llama_70b_q40 big model, why is there no output in the conversation.The large language model did not output effectively during the conversation.

[b4rtaz]

Hello, could you try to run the dllama inference mode to inspect if any token is processed?

[nodepend-code]

Hello, could you try to run the dllama inference mode to inspect if any token is processed?

The same situation occurs when using ./dllama inference, where the large language model does not output.

[b4rtaz]

Could you try with any smaller model? Maybe qwen3_8b_q40.

[nodepend-code]

Could you try with any smaller model? Maybe qwen3_8b_q40.

Hello! A smaller model can run normally, and I think using a large parameter model would be better. Is there a good way to adjust and achieve it?

[b4rtaz]

I think the problem may be for the llama architecture if a smaller qwen model works, could you try a bigger qwen model (qwen3_30b_a3b_q40)?

Could you dump the processor model (lscpu)?

[nodepend-code]

I think the problem may be for the llama architecture if a smaller qwen model works, could you try a bigger qwen model (qwen3_30b_a3b_q40)?

Could you dump the processor model (lscpu)?

Thanks for your reply. The (qwen3_30b_a3b_q40) model can also run normally. I would like to try using a larger model than (qwen3_30b_a3b_q40). My machine's CPU model is AMD RYZEN AI MAX+395 w/Radeon 8060S, why does the converted large parameter model not run properly? (e.g. deepseek_r1_distill_llama_70b_q40)

[b4rtaz]

Could you test a smaller llama model (llama3_1_8b_instruct_q40)?

[nodepend-code]

Could you test a smaller llama model (llama3_1_8b_instruct_q40)?

I just tested that (llama3_1_8b_intstruct_q40) can be used, but the 70B big language model I converted myself (llama3_3_70b_intstruct_q40, deepseek_r1_distill-Llama_70b_q40) cannot run properly on my GPU.

[b4rtaz]

Have you tried llama3_3_70b_instruct_q40 downloaded by launch.py? Maybe the tokenizer is broken.

[nodepend-code]

Have you tried llama3_3_70b_instruct_q40 downloaded by launch.py? Maybe the tokenizer is broken.

Thank you for your reply. I have re downloaded the llama3_3_70b_instruct_q40 model using launch.py. Running on GPU still has no output.I have found that super large models all have this problem (deepseek_r1_distill_Llama_70b_q40 is also the same).

[b4rtaz]

Could you check if all tests pass?

DLLAMA_VULKAN=1 make nn-vulkan-test && ./nn-vulkan-test

[nodepend-code]

Could you check if all tests pass?

DLLAMA_VULKAN=1 make nn-vulkan-test && ./nn-vulkan-test

All tests have been passed.

[nodepend-code]

Could you check if all tests pass?

DLLAMA_VULKAN=1 make nn-vulkan-test && ./nn-vulkan-test

All tests have been passed.

All DLLAMA_VULKAN tests have passed. There is no effective output when running the model llama3_3_70b_instruct_q40 on AMD GPU.May I ask what circumstances may lead to this situation.