Quantized KV Cache? #595
bkaruman
started this conversation in
New features / APIs
Quantized KV Cache?
#595
Replies: 2 comments
-
|
Any update on this ? |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Out of interest. What is quantized KV Caching? Which model is it for? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi! Is there a plan to support quantized KV Cache?
Beta Was this translation helpful? Give feedback.
All reactions