Request: How to use KV cache in this project? #282
Description
I don't find any examples or README about using kv cache with simulator in this project.
I followed this example https://github.com/DengHom/llm-d-inference-sim/tree/main/manifests/disaggregation to run pd disaggregation successfully, but I found it not used kv cache in this example.
What would you like to be added:
I want you can support something like examples or README to tell me how to use kv cache with simulator.
Why is this needed:
it's good for these people like me to learn how to use llm-d-inference-sim. I like this project !
Thank you all