Skip to content

Commit 32fb038

Browse files
committed
fix piecewise compilation in the ubatch wrapper
Signed-off-by: Sage Moore <[email protected]>
1 parent 9602070 commit 32fb038

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/compilation/ubatch_wrapper.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -253,7 +253,7 @@ def __call__(self, *args, **kwargs):
253253

254254
# If there's no ubatching, just run the runnable object
255255
if ubatch_slices is None:
256-
if cudagraph_runtime_mode is CUDAGraphMode.NONE:
256+
if cudagraph_runtime_mode in (CUDAGraphMode.NONE, CUDAGraphMode.PIECEWISE):
257257
return self.runnable(*args, **kwargs)
258258
else:
259259
assert self.cudagraph_wrapper is not None

0 commit comments

Comments
 (0)