Skip to content

[RFC]: Add dependencies field in StormService #1834

@lx1036

Description

@lx1036

🚀 Feature Description and Motivation

I deploy vLLM+LMCache+Mooncake with StormService in my production K8S like this:

apiVersion: orchestration.aibrix.ai/v1alpha1
kind: StormService
metadata:
  name: vllm-lmcache-mooncake
spec:
  replicas: 1
  updateStrategy:
    type: InPlaceUpdate
  stateful: true
  selector:
    matchLabels:
      app: vllm-lmcache-mooncake
  template:
    metadata:
      labels:
        app: vllm-lmcache-mooncake
    spec:
      roles:
        - name: mooncake-master
          replicas: 1
          stateful: true

        - name: mooncake-store
          replicas: 1
          stateful: true

       - name: vllm
          replicas: 1
          dependencies:  # new field
            - "mooncake-master"
          stateful: true

the vllm pod dependency on the mooncake-master pod, it needs mooncake-master pod starts firstly.

I found RBG https://github.com/sgl-project/rbg/blob/main/keps/74-mooncake-integration/README.md#deploy-standalone-inference-service-with-mooncake has the dependencies field.

it's best to add dependencies field at StormService.

Use Case

Deploy vLLM+Mooncake Store in K8S.

Proposed Solution

No response

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions