Skip to content

Commit 8675c7b

Browse files
JaredforRealJaredforReal
committed
combine dashboard and openwebui to k8s/o11y
Signed-off-by: JaredforReal <[email protected]>
1 parent 7c75aab commit 8675c7b

File tree

8 files changed

+928
-2
lines changed

8 files changed

+928
-2
lines changed

‎deploy/kubernetes/observability/README.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@ This guide adds a production-ready Prometheus + Grafana stack to the existing Se
1010
|--------------|---------|-----------|
1111
| Prometheus | Scrapes Semantic Router metrics and stores them with persistent retention | `prometheus/` (`rbac.yaml`, `configmap.yaml`, `deployment.yaml`, `pvc.yaml`, `service.yaml`)|
1212
| Grafana | Visualizes metrics using the bundled LLM Router dashboard and a pre-configured Prometheus datasource | `grafana/` (`secret.yaml`, `configmap-*.yaml`, `deployment.yaml`, `pvc.yaml`, `service.yaml`)|
13+
| Dashboard | Unified UI that links Router, Prometheus, and embeds Grafana; reads Router config | `dashboard/` (`configmap.yaml`, `deployment.yaml`, `service.yaml`)|
14+
| Open WebUI | Playground UI for interacting with the router via a Manifold Pipeline | `openwebui/` (`deployment.yaml`, `service.yaml`)|
15+
| Pipelines | Executes the `vllm_semantic_router_pipe.py` manifold for Open WebUI | `pipelines/deployment.yaml` (includes a ConfigMap with the pipeline code) |
1316
| Ingress (optional) | Exposes the UIs outside the cluster | `ingress.yaml`|
1417
| Dashboard provisioning | Automatically loads `deploy/llm-router-dashboard.json` into Grafana | `grafana/configmap-dashboard.yaml`|
1518

@@ -110,7 +113,7 @@ Verify pods:
110113
kubectl get pods -n vllm-semantic-router-system
111114
```
112115

113-
You should see `prometheus-...` and `grafana-...` pods in `Running` state.
116+
You should see `prometheus-...`, `grafana-...`, and `semantic-router-dashboard-...` pods in `Running` state.
114117

115118
### 5.3. Integration with the core deployment
116119

@@ -133,9 +136,11 @@ You should see `prometheus-...` and `grafana-...` pods in `Running` state.
133136
```bash
134137
kubectl port-forward svc/prometheus 9090:9090 -n vllm-semantic-router-system
135138
kubectl port-forward svc/grafana 3000:3000 -n vllm-semantic-router-system
139+
kubectl port-forward svc/semantic-router-dashboard 8700:80 -n vllm-semantic-router-system
140+
kubectl port-forward svc/openwebui 3001:8080 -n vllm-semantic-router-system
136141
```
137142

138-
Prometheus → http://localhost:9090, Grafana → http://localhost:3000
143+
Prometheus → http://localhost:9090, Grafana → http://localhost:3000, Dashboard → http://localhost:8700, Open WebUI → http://localhost:3001
139144

140145
- **Ingress (production)** – Customize `ingress.yaml` with real domains, TLS secrets, and your ingress class before applying. Replace `*.example.com` and configure HTTPS certificates via cert-manager or your provider.
141146

@@ -145,6 +150,7 @@ You should see `prometheus-...` and `grafana-...` pods in `Running` state.
145150
2. Query `rate(llm_model_completion_tokens_total[5m])` – should return data after traffic.
146151
3. Open Grafana, log in with the admin credentials, and confirm the **LLM Router Metrics** dashboard exists under the *Semantic Router* folder.
147152
4. Generate traffic to Semantic Router (classification or routing requests). Key panels should start populating:
153+
5.Playground: open Open WebUI (port-forward or ingress), select the `vllm-semantic-router/auto` model (from the Manifold pipeline), and send prompts. The Dashboard Monitoring page should reflect traffic, and the pipeline will display VSR decision headers inline.
148154
- Prompt Category counts
149155
- Token usage rate per model
150156
- Routing modifications between models

‎deploy/kubernetes/observability/dashboard/configmap.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
apiVersion: v1
2+
kind: ConfigMap
3+
metadata:
4+
name: semantic-router-dashboard-config
5+
labels:
6+
app: semantic-router-dashboard
7+
app.kubernetes.io/part-of: semantic-router
8+
app.kubernetes.io/component: observability
9+
data:
10+
TARGET_GRAFANA_URL: http://grafana.vllm-semantic-router-system.svc.cluster.local:3000
11+
TARGET_PROMETHEUS_URL: http://prometheus.vllm-semantic-router-system.svc.cluster.local:9090
12+
TARGET_ROUTER_API_URL: http://semantic-router.vllm-semantic-router-system.svc.cluster.local:8080
13+
TARGET_ROUTER_METRICS_URL: http://semantic-router-metrics.vllm-semantic-router-system.svc.cluster.local:9190/metrics
14+
TARGET_OPENWEBUI_URL: http://openwebui.vllm-semantic-router-system.svc.cluster.local:8080

‎deploy/kubernetes/observability/dashboard/deployment.yaml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: semantic-router-dashboard
5+
labels:
6+
app: semantic-router-dashboard
7+
spec:
8+
replicas: 1
9+
selector:
10+
matchLabels:
11+
app: semantic-router-dashboard
12+
template:
13+
metadata:
14+
labels:
15+
app: semantic-router-dashboard
16+
spec:
17+
containers:
18+
- name: dashboard
19+
image: ghcr.io/vllm-project/semantic-router/dashboard:latest
20+
imagePullPolicy: IfNotPresent
21+
args: ["-port=8700", "-static=/app/frontend", "-config=/app/config/config.yaml"]
22+
env:
23+
- name: TARGET_GRAFANA_URL
24+
valueFrom:
25+
configMapKeyRef:
26+
name: semantic-router-dashboard-config
27+
key: TARGET_GRAFANA_URL
28+
- name: TARGET_PROMETHEUS_URL
29+
valueFrom:
30+
configMapKeyRef:
31+
name: semantic-router-dashboard-config
32+
key: TARGET_PROMETHEUS_URL
33+
- name: TARGET_ROUTER_API_URL
34+
valueFrom:
35+
configMapKeyRef:
36+
name: semantic-router-dashboard-config
37+
key: TARGET_ROUTER_API_URL
38+
- name: TARGET_ROUTER_METRICS_URL
39+
valueFrom:
40+
configMapKeyRef:
41+
name: semantic-router-dashboard-config
42+
key: TARGET_ROUTER_METRICS_URL
43+
- name: TARGET_OPENWEBUI_URL
44+
valueFrom:
45+
configMapKeyRef:
46+
name: semantic-router-dashboard-config
47+
key: TARGET_OPENWEBUI_URL
48+
- name: ROUTER_CONFIG_PATH
49+
value: /app/config/config.yaml
50+
ports:
51+
- name: http
52+
containerPort: 8700
53+
volumeMounts:
54+
- name: router-config
55+
mountPath: /app/config
56+
readOnly: true
57+
volumes:
58+
- name: router-config
59+
configMap:
60+
name: semantic-router-config

‎deploy/kubernetes/observability/dashboard/service.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
apiVersion: v1
2+
kind: Service
3+
metadata:
4+
name: semantic-router-dashboard
5+
labels:
6+
app: semantic-router-dashboard
7+
spec:
8+
type: ClusterIP
9+
selector:
10+
app: semantic-router-dashboard
11+
ports:
12+
- name: http
13+
port: 80
14+
targetPort: http

‎deploy/kubernetes/observability/ingress.yaml‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,3 +51,59 @@ spec:
5151
name: prometheus
5252
port:
5353
name: http
54+
55+
---
56+
apiVersion: networking.k8s.io/v1
57+
kind: Ingress
58+
metadata:
59+
name: dashboard
60+
labels:
61+
app: semantic-router-dashboard
62+
annotations:
63+
kubernetes.io/ingress.class: nginx
64+
nginx.ingress.kubernetes.io/backend-protocol: HTTP
65+
nginx.ingress.kubernetes.io/ssl-redirect: "true"
66+
spec:
67+
tls:
68+
- hosts:
69+
- dashboard.example.com
70+
secretName: dashboard-tls
71+
rules:
72+
- host: dashboard.example.com
73+
http:
74+
paths:
75+
- path: /
76+
pathType: Prefix
77+
backend:
78+
service:
79+
name: semantic-router-dashboard
80+
port:
81+
name: http
82+
83+
---
84+
apiVersion: networking.k8s.io/v1
85+
kind: Ingress
86+
metadata:
87+
name: openwebui
88+
labels:
89+
app: openwebui
90+
annotations:
91+
kubernetes.io/ingress.class: nginx
92+
nginx.ingress.kubernetes.io/backend-protocol: HTTP
93+
nginx.ingress.kubernetes.io/ssl-redirect: "true"
94+
spec:
95+
tls:
96+
- hosts:
97+
- openwebui.example.com
98+
secretName: openwebui-tls
99+
rules:
100+
- host: openwebui.example.com
101+
http:
102+
paths:
103+
- path: /
104+
pathType: Prefix
105+
backend:
106+
service:
107+
name: openwebui
108+
port:
109+
name: http

‎deploy/kubernetes/observability/kustomization.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,9 @@ resources:
1919
- grafana/configmap-dashboard.yaml
2020
- grafana/deployment.yaml
2121
- grafana/service.yaml
22+
- dashboard/configmap.yaml
23+
- dashboard/deployment.yaml
24+
- dashboard/service.yaml
25+
- pipelines/deployment.yaml
26+
- openwebui/deployment.yaml
2227
- ingress.yaml

‎deploy/kubernetes/observability/openwebui/deployment.yaml‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: openwebui
5+
labels:
6+
app: openwebui
7+
spec:
8+
replicas: 1
9+
selector:
10+
matchLabels:
11+
app: openwebui
12+
template:
13+
metadata:
14+
labels:
15+
app: openwebui
16+
spec:
17+
containers:
18+
- name: openwebui
19+
image: ghcr.io/open-webui/open-webui:main
20+
imagePullPolicy: IfNotPresent
21+
ports:
22+
- name: http
23+
containerPort: 8080
24+
env:
25+
- name: WEBUI_NAME
26+
value: "Open WebUI"
27+
- name: OPENAI_API_BASE_URL
28+
value: "http://openwebui-pipelines:9099"
29+
- name: OPENAI_API_KEY
30+
value: "0p3n-w3bu!"
31+
volumeMounts:
32+
- name: data
33+
mountPath: /app/backend/data
34+
resources:
35+
requests:
36+
cpu: 100m
37+
memory: 256Mi
38+
limits:
39+
cpu: 500m
40+
memory: 1Gi
41+
volumes:
42+
- name: data
43+
emptyDir: {}
44+
---
45+
apiVersion: v1
46+
kind: Service
47+
metadata:
48+
name: openwebui
49+
labels:
50+
app: openwebui
51+
spec:
52+
selector:
53+
app: openwebui
54+
ports:
55+
- name: http
56+
port: 8080
57+
targetPort: http
58+
type: ClusterIP

0 commit comments

Comments
 (0)