Commit 8d7a0d7

committed

Introduce offloading connector

This commit introduces a new OffloadingConnector for offloading blocks of KV data via a generic interface. Signed-off-by: Or Ozeri <[email protected]>

1 parent 1d238f2 commit 8d7a0d7Copy full SHA for 8d7a0d7

6 files changed

+1104

-1

lines changed

tests/v1/kv_connector/unit
- test_offloading_connector.py
- utils.py
vllm
- distributed/kv_transfer/kv_connector
  - factory.py
  - v1
    - offloading_connector.py
- v1/offloading
  - factory.py
  - spec.py

6 files changed

+1104

-1

lines changed

Comments

(0)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Commit 8d7a0d7

6 files changed

6 files changed

Uh oh!

File tree

6 files changed

6 files changed

0 commit comments