feat: add Grafana+Prometheus in k8s #294

JaredforReal · 2025-09-30T04:32:05Z

What type of PR is this?
feat: add Grafana+Prometheus in k8s

What this PR does / why we need it:
Add deployment guide in deploy/kubernetes/observability/README.md.
Add config and manifest of Grafana in deploy/kubernetes/observability/grafana/
Add config and manifest of Prometheus in deploy/kubernetes/observability/prometheus/
Add deployment config in deploy/kubernetes/observability/kustomization.yaml
Add optional HTTPS ingress examples in deploy/kubernetes/observability/ingress.yaml
Update docs in website/docs/tutorial/observability/observability.md
Update the random UID to "prometheus" in deploy/kubernetes/llm-router-dashboard.json

Which issue(s) this PR fixes:
Fixes #279

Signed-off-by: JaredforReal <[email protected]>

netlify · 2025-09-30T04:32:10Z

✅ Deploy Preview for vllm-semantic-router ready!

Name	Link
🔨 Latest commit	`790c919`
🔍 Latest deploy log	https://app.netlify.com/projects/vllm-semantic-router/deploys/68dcac79d26e0300081ded17
😎 Deploy Preview	https://deploy-preview-294--vllm-semantic-router.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

github-actions · 2025-09-30T04:34:24Z

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 `deploy`

Owners: @rootfs, @Xunzhuo
Files changed:

deploy/kubernetes/observability/README.md
deploy/kubernetes/observability/grafana/configmap-dashboard.yaml
deploy/kubernetes/observability/grafana/configmap-provisioning.yaml
deploy/kubernetes/observability/grafana/deployment.yaml
deploy/kubernetes/observability/grafana/pvc.yaml
deploy/kubernetes/observability/grafana/secret.yaml
deploy/kubernetes/observability/grafana/service.yaml
deploy/kubernetes/observability/ingress.yaml
deploy/kubernetes/observability/kustomization.yaml
deploy/kubernetes/observability/prometheus/configmap.yaml
deploy/kubernetes/observability/prometheus/deployment.yaml
deploy/kubernetes/observability/prometheus/pvc.yaml
deploy/kubernetes/observability/prometheus/rbac.yaml
deploy/kubernetes/observability/prometheus/service.yaml
deploy/llm-router-dashboard.json

📁 `website`

Owners: @Xunzhuo
Files changed:

website/docs/troubleshooting/network-tips.md
website/docs/tutorials/observability/observability.md

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

JaredforReal · 2025-09-30T04:36:01Z

More network troubleshooting will be updated in docs/troubleshooting/network-tips.md

Xunzhuo · 2025-09-30T11:16:33Z

deploy/kubernetes/observability/README.md

@@ -0,0 +1,203 @@
+# Semantic Router Observability on Kubernetes
+
+This guide adds a production-ready Prometheus + Grafana stack to the existing Semantic Router Kubernetes deployment. It includes manifests for collectors, dashboards, data sources, RBAC, and ingress so you can monitor routing performance in any cluster.


Can move this to tutorials in website, https://vllm-semantic-router.com/docs/tutorials/observability/ ?

Sure，scheduled

Signed-off-by: JaredforReal <[email protected]>

JaredforReal · 2025-09-30T11:37:43Z

Should I keep the README.md in deploy/kubernetes/observability/? Since I have merged it into docs/tutorialobservability/observability.md.
Should I split the docs/tutorialobservability/observability.md to /docker-compose.md and kubernetes.md?
@Xunzhuo WDYT?

rootfs · 2025-09-30T12:45:41Z

deploy/kubernetes/observability/grafana/configmap-dashboard.yaml

@@ -0,0 +1,652 @@
+apiVersion: v1


is this the same dashboard as in https://github.com/vllm-project/semantic-router/tree/main/config/grafana? Is there a way we can consolidate them in a follow up PR?

We could consolidate them — it would add some complexity (e.g. conditional processing with Kustomize overlays), but overall make maintenance easier.

rootfs · 2025-09-30T12:46:06Z

deploy/kubernetes/observability/grafana/configmap-provisioning.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1


same issue as above

rootfs · 2025-09-30T12:46:43Z

deploy/kubernetes/observability/grafana/deployment.yaml

+    spec:
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 472


is 472 a magic number?

As seen in lines 20 and 21 in /prometheus/deployment.yaml, Prometheus runs as UID/GID 65534 (nobody), while Grafana runs as UID/GID 472 (a dedicated Grafana user) to avoid running processes as root and to enhance container security, particularly when mounting volumes. Maybe I can add a short comment here.

got it, thanks! Adding a comment is helpful.

rootfs · 2025-09-30T12:48:08Z

@fcanogab can you take a look? Thanks

Signed-off-by: JaredforReal <[email protected]>

JaredforReal · 2025-10-01T04:25:23Z

More network troubleshooting will be updated in docs/troubleshooting/network-tips.md

done! @Xunzhuo @rootfs it's ready for review now. Thanks!

fcanogab · 2025-10-01T08:30:58Z

@fcanogab can you take a look? Thanks

LGTM. Thanks.

* feat: add Grafava+Prometheus in k8s Signed-off-by: JaredforReal <[email protected]> * Update docs of observability k8s part Signed-off-by: JaredforReal <[email protected]> * get rig of redudent part in doc Signed-off-by: JaredforReal <[email protected]> * add comments of 472 and 65534 Signed-off-by: JaredforReal <[email protected]> * add network tips of k8s Signed-off-by: JaredforReal <[email protected]> * update uid in dashboard Signed-off-by: JaredforReal <[email protected]> --------- Signed-off-by: JaredforReal <[email protected]> Co-authored-by: Huamin Chen <[email protected]> Signed-off-by: liuhy <[email protected]>

feat: add Grafava+Prometheus in k8s

0665cbf

Signed-off-by: JaredforReal <[email protected]>

Merge branch 'main' into obser

cb70540

github-actions bot assigned rootfs and Xunzhuo Sep 30, 2025

Xunzhuo reviewed Sep 30, 2025

View reviewed changes

JaredforReal added 2 commits September 30, 2025 19:27

Merge branch 'main' into obser

4096e4e

Update docs of observability k8s part

cedd13e

Signed-off-by: JaredforReal <[email protected]>

JaredforReal marked this pull request as ready for review September 30, 2025 11:31

JaredforReal requested a review from rootfs as a code owner September 30, 2025 11:31

get rig of redudent part in doc

9405f47

Signed-off-by: JaredforReal <[email protected]>

rootfs reviewed Sep 30, 2025

View reviewed changes

deploy/kubernetes/observability/grafana/configmap-provisioning.yaml

@@ -0,0 +1,30 @@

apiVersion: v1

Copy link

Collaborator

rootfs Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same issue as above

rootfs reviewed Sep 30, 2025

View reviewed changes

rootfs previously approved these changes Sep 30, 2025

View reviewed changes

Merge branch 'main' into obser

abad95a

JaredforReal added 2 commits October 1, 2025 11:42

Merge branch 'main' into obser

ebbce20

add comments of 472 and 65534

5249c10

Signed-off-by: JaredforReal <[email protected]>

JaredforReal dismissed rootfs’s stale review via 5249c10 October 1, 2025 03:43

JaredforReal added 2 commits October 1, 2025 11:49

add network tips of k8s

496a8bd

Signed-off-by: JaredforReal <[email protected]>

update uid in dashboard

790c919

Signed-off-by: JaredforReal <[email protected]>

rootfs approved these changes Oct 1, 2025

View reviewed changes

rootfs merged commit dff45b3 into vllm-project:main Oct 1, 2025
9 checks passed

JaredforReal deleted the obser branch October 2, 2025 01:12

		@@ -0,0 +1,203 @@
		# Semantic Router Observability on Kubernetes

		This guide adds a production-ready Prometheus + Grafana stack to the existing Semantic Router Kubernetes deployment. It includes manifests for collectors, dashboards, data sources, RBAC, and ingress so you can monitor routing performance in any cluster.

feat: add Grafana+Prometheus in k8s #294

feat: add Grafana+Prometheus in k8s #294

Uh oh!

Conversation

JaredforReal commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for vllm-semantic-router ready!

Uh oh!

github-actions bot commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

👥 vLLM Semantic Team Notification

📁 deploy

📁 website

🎉 Thanks for your contributions!

Uh oh!

JaredforReal commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Xunzhuo Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

JaredforReal Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JaredforReal commented Sep 30, 2025

Uh oh!

rootfs Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

JaredforReal Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

rootfs Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

rootfs Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

JaredforReal Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

rootfs Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

rootfs commented Sep 30, 2025

Uh oh!

JaredforReal commented Oct 1, 2025

Uh oh!

fcanogab commented Oct 1, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

JaredforReal commented Sep 30, 2025 •

edited

Loading

netlify bot commented Sep 30, 2025 •

edited

Loading

github-actions bot commented Sep 30, 2025 •

edited

Loading

📁 `deploy`

📁 `website`

JaredforReal commented Sep 30, 2025 •

edited

Loading

JaredforReal Sep 30, 2025 •

edited

Loading