Eval bug: unknown model architecture: 'qwen35moe'

[happypatwu]

Name and Version

[2602.01 version]

Operating systems

Linux

GGML backends

CUDA

Hardware

MI50 32G

Models

No response

Problem description & steps to reproduce

llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35moe'
llama_model_load_from_file_impl: failed to load model
common_init_from_params: failed to load model '/models/unsloth/Qwen3.5-35B-A3B-Q4_K_M.gguf'
srv load_model: failed to load model, '/models/unsloth/Qwen3.5-35B-A3B-Q4_K_M.gguf'
srv operator(): operator(): cleaning up before exit...
main: exiting due to model loading error

First Bad Commit

No response

Relevant log output

llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35moe'
llama_model_load_from_file_impl: failed to load model
common_init_from_params: failed to load model '/models/unsloth/Qwen3.5-35B-A3B-Q4_K_M.gguf'
srv load_model: failed to load model, '/models/unsloth/Qwen3.5-35B-A3B-Q4_K_M.gguf'
srv operator(): operator(): cleaning up before exit...
main: exiting due to model loading error

[Xande-p]

Not exactly a bug, but support for the new qwen35 and qwen35moe architectures will be a very much welcomed addition! Main project was segfaulting already on qwen3next, now also on qwen35 and qwen35moe, not sure if gfx906 is in their plans at all for the future.

[gurkburk76]

would be great with an update, assuming mainline hasen't caught up with speedier build for gfx906 🥇

[schlick7]

Theres 2 issues here.
1- Llama.cpp hasn't been updated in this repo to support the new qwen35.
2- Qwen3next and qwen35 will segfault if using Rocm 7.x as changes need to be made to llama.cpp (seems to work with rocm 6.3)

[Xande-p]

This fork was working with ROCm 7.2 even after main stopped working, so maybe we can still use 7.2, but also it's not a big deal to go back to 6.3 if needed.
But a heads up: today's update of main caused 20% drop in TG for GFX906 in ROCm (no changes using Vulkan), just a FYI if iacop updates this fork to latest llama. Also, qwen3.5 is running about 20% faster on ik_llama compared do old main (40% faster than new main), so maybe take a look on what ik is doing as well?
Anyway, just suggestions, I do appreciate iacop work! 👍

[slavap]

See mixa3607/ML-gfx906#10
Llama 8209 has no 20% drop in TG