Skip to content

Commit 15f0b6a

Browse files
authored
fix README diagram (#75)
Signed-off-by: Maroon Ayoub <[email protected]>
1 parent 9359a94 commit 15f0b6a

File tree

1 file changed

+15
-15
lines changed

1 file changed

+15
-15
lines changed

README.md

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -47,24 +47,24 @@ graph TD
4747
G[...]
4848
end
4949
50-
A -->|"1. Score(prompt, pods)"| B
51-
B -->|2. Query Index| C
52-
B -->|3. Return Scores| A
50+
A -->|"(1) Score(prompt, pods)"| B
51+
B -->|"(2) Query Index"| C
52+
B -->|"(3) Return Scores"| A
5353
54-
E -->|A. Emit KVEvents| D
55-
F -->|A. Emit KVEvents| D
56-
D -->|B. Update Index| C
54+
E -->|"(A) Emit KVEvents"| D
55+
F -->|"(A) Emit KVEvents"| D
56+
D -->|"(B) Update Index"| C
5757
```
58-
_Note: 1-3 represent the Read Path for scoring pods, while A-B represent the Write Path for ingesting KVEvents._
58+
**Read Path:**
59+
- (1) **Scoring Request**: A scheduler asks the **KVCache Indexer** to score a set of pods for a given prompt
60+
- (2) **Index Query**: The indexer calculates the necessary KV-block keys from the prompt and queries the **KV-Block Index** to see which pods have those blocks
61+
- (3) **Return Scores**: The indexer returns a map of pods and their corresponding KV-cache-hit scores to the scheduler
5962

60-
1. **Scoring Request**: A scheduler asks the **KVCache Indexer** to score a set of pods for a given prompt
61-
2. **Index Query**: The indexer calculates the necessary KV-block keys from the prompt and queries the **KV-Block Index** to see which pods have those blocks
62-
3. **Return Scores**: The indexer returns a map of pods and their corresponding KV-cache-hit scores to the scheduler
63-
4. **Event Ingestion**: As vLLM pods create or evict KV-blocks, they emit `KVEvents` containing metadata about these changes
64-
5. **Index Update**: The **Event Subscriber** consumes these events and updates the **KV-Block Index** in near-real-time
63+
**Write Path:**
64+
- (A) **Event Ingestion**: As vLLM pods create or evict KV-blocks, they emit `KVEvents` containing metadata about these changes
65+
- (B) **Index Update**: The **Event Subscriber** consumes these events and updates the **KV-Block Index** in near-real-time
6566

66-
* For a more detailed breakdown, please see the high-level [Architecture Document](docs/architecture.md).
67-
* For configuration details, see the [Configuration Document](docs/configuration.md).
67+
> For a more detailed breakdown, please see the high-level [Architecture](docs/architecture.md) and the [Configuration](docs/configuration.md) docs.
6868
6969
-----
7070

@@ -75,4 +75,4 @@ _Note: 1-3 represent the Read Path for scoring pods, while A-B represent the Wri
7575
* [**KVCache Aware Scorer**](examples/kv_cache_aware_scorer/README.md):
7676
A reference implementation of how to integrate the `kvcache.Indexer` into a scheduler like the `llm-d-inference-scheduler`
7777
* [**KV-Events**](examples/kv_events/README.md):
78-
Demonstrates how the KV-Cache Manager handles KV-Events through both an offline example with a dummy ZMQ publisher and an online example using a vLLM Helm chart.
78+
Demonstrates how the KV-Cache Manager handles KV-Events through both an offline example with a dummy ZMQ publisher and an online example using a vLLM Helm chart.

0 commit comments

Comments
 (0)