Skip to content

Can't run Qwen3-MoE models on 8 nodes #270

@D-i-t-gh

Description

@D-i-t-gh

Hi,

I can run Qwen3-MoE models (nKvHeads: 4) on 4 nodes without problems and it works great - thank you very much!

If I try to run the same model on 8 nodes I get this error:
🚨 Critical error: This version does not support more nodes than the number of KV heads in the model

It seems there can't be more than nKvHeads nodes. Is there any way to run Qwen3-MoE models on 8 nodes?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions