Skip to content

Commit c6f93dd

Browse files
committed
add monkey patch for kvcomp on device
1 parent 0ab52c2 commit c6f93dd

File tree

3 files changed

+536
-29
lines changed

3 files changed

+536
-29
lines changed

ucm/__init__.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
from ucm.integration.vllm.ucm_connector import UCMConnector
2+
3+
try:
4+
from ucm.integration.vllm.patch.apply_patch import ensure_patches_applied
5+
6+
ensure_patches_applied()
7+
except Exception as e:
8+
# Don't fail if patches can't be applied - might be running in environment without vLLM
9+
import warnings
10+
11+
warnings.warn(
12+
f"Failed to apply vLLM patches: {e}. "
13+
f"If you're using vLLM, ensure it's installed and patches are compatible."
14+
)
15+
16+
__all__ = ["UCMConnector"]

0 commit comments

Comments
 (0)