-
Notifications
You must be signed in to change notification settings - Fork 500
Open
Description
🚀 Feature Description and Motivation
I deploy vLLM+LMCache+Mooncake with StormService in my production K8S like this:
apiVersion: orchestration.aibrix.ai/v1alpha1
kind: StormService
metadata:
name: vllm-lmcache-mooncake
spec:
replicas: 1
updateStrategy:
type: InPlaceUpdate
stateful: true
selector:
matchLabels:
app: vllm-lmcache-mooncake
template:
metadata:
labels:
app: vllm-lmcache-mooncake
spec:
roles:
- name: mooncake-master
replicas: 1
stateful: true
- name: mooncake-store
replicas: 1
stateful: true
- name: vllm
replicas: 1
dependencies: # new field
- "mooncake-master"
stateful: true
the vllm pod dependency on the mooncake-master pod, it needs mooncake-master pod starts firstly.
I found RBG https://github.com/sgl-project/rbg/blob/main/keps/74-mooncake-integration/README.md#deploy-standalone-inference-service-with-mooncake has the dependencies field.
it's best to add dependencies field at StormService.
Use Case
Deploy vLLM+Mooncake Store in K8S.
Proposed Solution
No response
Metadata
Metadata
Assignees
Labels
No labels