Skip to content

Commit a704967

Browse files
authored
[V0.9.1][BugFix] Fix the bug in decoraotor patch (#2199)
### What this PR does / why we need it? Fix the bug in the decorator patch, which would have a negative impact on performance with mini batch. Now fix it by enabling this patch just when engine_v1 and flashcomm_v1 enabled together. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? CI passed with new added/existing test. Related: #2191 (comment) Signed-off-by: rjg-lyh <[email protected]>
1 parent 2b97c69 commit a704967

File tree

2 files changed

+6
-2
lines changed

2 files changed

+6
-2
lines changed

vllm_ascend/__init__.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,14 @@ def register():
2525
def register_model():
2626
# fix pytorch schema check error, remove this line after pytorch
2727
# is upgraded to 2.7.0
28+
import vllm.envs as envs
29+
2830
import vllm_ascend.patch.worker.patch_common.patch_utils # noqa: F401
2931

3032
from .models import register_model
3133

32-
import vllm_ascend.patch.platform.patch_0_9_1.patch_decorator # isort: skip # noqa: F401
34+
import vllm_ascend.envs as envs_ascend # isort: skip # noqa: F401
35+
if envs.VLLM_USE_V1 and \
36+
envs_ascend.VLLM_ASCEND_ENABLE_FLASHCOMM != 0:
37+
import vllm_ascend.patch.platform.patch_0_9_1.patch_decorator # isort: skip # noqa: F401
3338
register_model()

vllm_ascend/patch/platform/patch_0_9_1/__init__.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,3 @@
1919
# patch files.
2020
import vllm_ascend.patch.worker.patch_common.patch_utils # noqa isort:skip
2121
import vllm_ascend.patch.platform.patch_0_9_1.patch_cache_manager # noqa
22-
import vllm_ascend.patch.platform.patch_0_9_1.patch_decorator # noqa

0 commit comments

Comments
 (0)