Skip to content

Commit 6d2bc08

Browse files
committed
Add execute_begin method and reset is_tensor_computed to be false during each step
1 parent d7f4d32 commit 6d2bc08

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

ucm/sparse/kvcomp/kvcomp_hbm.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
from vllm.forward_context import ForwardContext
1212
from vllm.v1.attention.backends.mla.common import MLACommonMetadata
1313
from vllm.v1.request import Request, RequestStatus
14+
from vllm.v1.core.sched.output import SchedulerOutput
1415

1516
from ucm.logger import init_logger
1617
from ucm.sparse.base import (
@@ -459,6 +460,9 @@ def request_finished_in_scheduler(self, request_id: Union[int, str]):
459460
"""
460461
pass
461462

463+
def execute_begin(self, scheduler_output: SchedulerOutput):
464+
self.is_tensor_computed = False
465+
462466
def estimate_num_slots_sparsed(self, request: Request) -> int:
463467
return INVALID_SLOT
464468

0 commit comments

Comments
 (0)