[Bug]: No MTP support for Qwen3 Next

[nutriarch]

System Info

There currently is not any support for MTP like DeepSeek has for Qwen3 next

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Attempt to run Qwen3 Next like the example but with MTP

Expected behavior

Support like DeepSeek has

actual behavior

Super slow inference due to the fact that there's no MTP speculative decoding support for Qwen3 Next

additional notes

TensorRT-LLM/tensorrt_llm/_torch/models/modeling_deepseekv3.py

Line 823 in d6b741d

class DeepseekV3MTP(DeepseekV3DecoderLayer):

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!