Commit 9d2e0b9

committed

Introduce offloading connector

This commit introduces a new OffloadingConnector for offloading blocks of KV data via a generic interface. Signed-off-by: Or Ozeri <[email protected]>

1 parent ebc09b3 commit 9d2e0b9Copy full SHA for 9d2e0b9

6 files changed

+1104

-1

lines changed

tests/v1/kv_connector/unit
- test_offloading_connector.py
- utils.py
vllm
- distributed/kv_transfer/kv_connector
  - factory.py
  - v1
    - offloading_connector.py
- v1/offloading
  - factory.py
  - spec.py

6 files changed

+1104

-1

lines changed

Comments

(0)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Commit 9d2e0b9

6 files changed

6 files changed

Uh oh!

File tree

6 files changed

6 files changed

0 commit comments