Fix qwen3 vl moe execution failure#992
Fix qwen3 vl moe execution failure#992shepark wants to merge 4 commits intovllm-project:releases/v0.15.1from
Conversation
shepark
requested review from
mgawarkiewicz-intel,
piotrbocian and
wpyszka
as code owners
shepark
force-pushed
the
shepark/fix_qwen3_vl_moe_failure
branch
from
4198669 to
6dc9601
Compare
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
shepark
force-pushed
the
shepark/fix_qwen3_vl_moe_failure
branch
from
6dc9601 to
5afa931
Compare
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
shepark
force-pushed
the
shepark/fix_qwen3_vl_moe_failure
branch
from
5afa931 to
1846288
Compare
vllm_gaudi/models/qwen3_moe.py
Outdated
| # 1st changes: overide only forward func to support 3d tensor input | ||
| # convert (B, S, H) -> (B*S, H) | ||
| # 2nd changes: corresponding changes to upstream (8edaf385) | ||
| # SharedFusedMoE support |
There was a problem hiding this comment.
NIT: This dev comment looks PR specific, can we make it more universal and as a docstring?
class HpuQwen3MoeSparseMoeBlock(UpstreamQwen3MoeSparseMoeBlock): """Override forward to handle 3D tensor input (B,S,H) -> (B*S,H) and SharedFusedMoE tuple returns."""
There was a problem hiding this comment.
NIT: This dev comment looks PR specific, can we make it more universal and as a docstring?
class HpuQwen3MoeSparseMoeBlock(UpstreamQwen3MoeSparseMoeBlock): """Override forward to handle 3D tensor input (B,S,H) -> (B*S,H) and SharedFusedMoE tuple returns."""
Thank you for the review. I will update this.
shepark
force-pushed
the
shepark/fix_qwen3_vl_moe_failure
branch
from
1846288 to
fb61400
Compare
Signed-off-by: Seunghyuk Park <separk@habana.ai>
shepark
force-pushed
the
shepark/fix_qwen3_vl_moe_failure
branch
from
fb61400 to
3c68bf1
Compare
4 participants
These are the changes for the qwen3 vl moe model corresponding to 3a7195e
and to the upstream (v0.15.1) change (vllm-project/vllm@8edaf38)