Commit 4b24d03

committed

Introduce offloading connector

This commit introduces a new OffloadingConnector for offloading blocks of KV data via a generic interface. Signed-off-by: Or Ozeri <[email protected]>

1 parent e7b1232 commit 4b24d03Copy full SHA for 4b24d03

6 files changed

+1065

-1

lines changed

tests/v1/kv_connector/unit
- test_offloading_connector.py
- utils.py
vllm
- distributed/kv_transfer/kv_connector
  - factory.py
  - v1
    - offloading_connector.py
- v1/offloading
  - factory.py
  - spec.py

6 files changed

+1065

-1

lines changed

Comments

(0)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Commit 4b24d03

6 files changed

6 files changed

Uh oh!

File tree

6 files changed

6 files changed

0 commit comments