Skip to content

Commit 358ba68

Browse files
authored
[main][bugfix] Fix MatmulNZ format bug on some machines (#2549)
### What this PR does / why we need it? This PR fixes the bug on some machines where quantmatmul failed to run with the NZ format. The change ensures proper execution under the expected data layout. ### How was this patch tested? CI passed with existing test. - vLLM version: v0.10.1.1 - vLLM main: vllm-project/vllm@b5d34af Signed-off-by: rjg-lyh <[email protected]>
1 parent 042605f commit 358ba68

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

vllm_ascend/worker/model_runner_v1.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,9 @@
112112

113113
import vllm_ascend.envs as envs_ascend
114114

115+
# if true, allow tensor initialization and casting with internal format (e.g., NZ)
116+
torch.npu.config.allow_internal_format = True
117+
115118
if is_310p():
116119
torch_npu.npu.set_compile_mode(jit_compile=False)
117120
ACL_FORMAT = ACL_FORMAT_FRACTAL_NZ

0 commit comments

Comments
 (0)