Skip to content

Commit 40a7c78

Browse files
committed
add monkey patch for kvcomp on device
1 parent 6899198 commit 40a7c78

File tree

3 files changed

+467
-19
lines changed

3 files changed

+467
-19
lines changed

ucm/__init__.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
from ucm.integration.vllm.ucm_connector import UCMConnector
2+
3+
try:
4+
from ucm.integration.vllm.patch.apply_patch import ensure_patches_applied
5+
6+
ensure_patches_applied()
7+
except Exception as e:
8+
# Don't fail if patches can't be applied - might be running in environment without vLLM
9+
import warnings
10+
11+
warnings.warn(
12+
f"Failed to apply vLLM patches: {e}. "
13+
f"If you're using vLLM, ensure it's installed and patches are compatible."
14+
)
15+
16+
__all__ = ["UCMConnector"]

0 commit comments

Comments
 (0)