[v0.1] Environment:Container and k8s deployment readiness

[rootfs]

Acceptance

• K8s manifests with model init container, health/readiness probes, resource limits, and metrics endpoints; documented deployment flow.
• Operator that uses LLM model eval to generate config yaml and startup the k8s deployment

[yuluo-yx]

hi @rootfs . for this issue.

i found https://github.com/vllm-project/semantic-router/tree/main/docker and https://github.com/vllm-project/semantic-router/tree/main/deploy/kubernetes.

from this point of view, this issue should be completed, right?

[rootfs]

@yuluo-yx the 2nd item is not started yet. If you are interested, getting the operator is a good start.

[yuluo-yx]

@yuluo-yx the 2nd item is not started yet. If you are interested, getting the operator is a good start.

yep, I want to try. First, I want to know what exactly to do?

from the description, I understand I need to write a Kubernetes operator.

The operator uses a llm to generate the Kubernetes cluster resource (CRD) configuration required by the application. And then publish it?

my understanding may be a little wrong. In fact, I don't understand the meaning of the sentence uses LLM model eval to generate config yaml. 🥲

[yuluo-yx]

Is this a duplicate of #98 ? @rootfs

[rootfs]

@yuluo-yx looks so to me too. Closing

[rootfs]

Closing in favor of #98