docs: add remote backend Valkey option for KV offloading + Feast#703
Open
edlng wants to merge 1 commit into
Open
docs: add remote backend Valkey option for KV offloading + Feast#703edlng wants to merge 1 commit into
edlng wants to merge 1 commit into
Conversation
✅ Deploy Preview for elastic-nobel-0aef7a ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
… Feast Signed-off-by: Edward Liang <edward.liang@improving.com>
edlng
force-pushed
the
valkey-kvcache-offloading-docs
branch
from
6f2b81a to
6c450ed
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a "Using Valkey as Remote Backend" section to the KV-cache offloading guide (right next to the existing Redis section) and a "Using Valkey instead of Redis" section to the Feast feature store guide. Both changes are mirrored into
versioned_docs/version-0.18/so the current and 0.18 docs stay in sync.Valkey already works as a drop-in Redis replacement for both of these: LMCache's
ValkeyConnectorspeaks the same wire protocol, and Feast's Redis online-store client doesn't care whether it's talking to Redis or Valkey on the other end.Issue
Related: #701
Changes
docs/model-serving/generative-inference/kvcache-offloading/kvcache-offloading.md-- new Valkey section with ConfigMap/Deployment/Service YAML, plus a link to the paired sample manifest in kserve/kserve.docs/model-serving/predictive-inference/transformers/feast-feature-store/feast-feature-store.md-- new section pointing at thevalkey.yamlsample and noting the one thing that needs manual attention: the feature-store-initializer entrypoint script hardcodesredis-servicein its connection string, so anyone using the Valkey sample needs to update that before building the initializer image.versioned_docs/version-0.18/...(both files) -- identical copies of the above, kept byte-for-byte in sync with thedocs/versions.Tests
No automated tests here since this is a doc change, but I checked every link by hand: every
kserve.github.io/website/...URL resolves with a livecurl -I, and I diffed thedocs/andversioned_docs/version-0.18/copies against each other to confirm they match.The companion kserve/kserve PR's LMCache+Valkey sample was validated end-to-end on a GPU instance (g5.2xlarge, A10G): vLLM + LMCache successfully stored KV cache chunks to a live Valkey container and retrieved them on subsequent requests (confirmed via
valkey-cli KEYSand LMCache'shit tokenslogs).One thing worth flagging: the two links to the sample files in
kserve/kserve(llmisvc-lmcache-valkey.yamlandvalkey.yaml) point at files that don't exist on that repo'smasteryet -- they're on a branch in a companion PR. This PR is meant to merge after that one lands, at which point the links resolve. If review order ends up flipped, ping me and I'll swap those to branch URLs in the meantime.Additional context
Companion PR: kserve/kserve (adds
docs/samples/llmisvc/lmcache-valkey-offloading/,docs/samples/llmisvc/precise-prefix-kv-cache-routing/valkey-shared-index/, anddocs/samples/v1beta1/transformer/feast/valkey.yaml).Currently, some of the links to
kserve/kservemaster won't resolve until the PR on that repository has been merged.