Skip to content

Commit f3c66b0

Browse files
JaredforRealJaredforReal
authored
feat: set up Grafana and Prometheus for Observability and Monitoring (#222)
* add Grafana and Prometheus & docs for setting up Signed-off-by: JaredforReal <[email protected]> * focus on MVP with local docker compose Signed-off-by: JaredforReal <[email protected]> * refactor observability.md Signed-off-by: JaredforReal <[email protected]> --------- Signed-off-by: JaredforReal <[email protected]>
1 parent 0a08e67 commit f3c66b0

File tree

6 files changed

+121
-0
lines changed

6 files changed

+121
-0
lines changed

‎config/grafana/dashboards.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: 1
2+
providers:
3+
- name: LLM Router Dashboards
4+
orgId: 1
5+
folder: "LLM Router"
6+
type: file
7+
disableDeletion: false
8+
allowUiUpdates: true
9+
options:
10+
path: /etc/grafana/provisioning/dashboards

‎config/grafana/datasource.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
apiVersion: 1
2+
datasources:
3+
- name: Prometheus
4+
type: prometheus
5+
access: proxy
6+
url: http://prometheus:9090
7+
isDefault: true

‎config/prometheus.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
global:
2+
scrape_interval: 10s
3+
evaluation_interval: 10s
4+
5+
scrape_configs:
6+
# Semantic Router
7+
- job_name: semantic-router
8+
metrics_path: /metrics
9+
static_configs:
10+
- targets: ["semantic-router:9190"]
11+
labels:
12+
service: semantic-router
13+
env: dev
14+
15+
# Optional: Envoy
16+
- job_name: envoy
17+
metrics_path: /stats/prometheus
18+
static_configs:
19+
- targets: ["envoy-proxy:19000"]
20+
labels:
21+
service: envoy
22+
env: dev

‎docker-compose.yml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,35 @@ services:
6464
retries: 5
6565
start_period: 5s
6666

67+
# Prometheus and Grafana for observability
68+
prometheus:
69+
image: prom/prometheus:v2.53.0
70+
container_name: prometheus
71+
volumes:
72+
- ./config/prometheus.yml:/etc/prometheus/prometheus.yml:ro
73+
command:
74+
- --config.file=/etc/prometheus/prometheus.yml
75+
- --storage.tsdb.retention.time=15d
76+
ports:
77+
- "9090:9090"
78+
networks:
79+
- semantic-network
80+
81+
grafana:
82+
image: grafana/grafana:11.5.1
83+
container_name: grafana
84+
environment:
85+
- GF_SECURITY_ADMIN_USER=admin
86+
- GF_SECURITY_ADMIN_PASSWORD=admin
87+
ports:
88+
- "3000:3000"
89+
volumes:
90+
- ./config/grafana/datasource.yml:/etc/grafana/provisioning/datasources/datasource.yml:ro
91+
- ./config/grafana/dashboards.yml:/etc/grafana/provisioning/dashboards/dashboards.yml:ro
92+
- ./deploy/llm-router-dashboard.json:/etc/grafana/provisioning/dashboards/llm-router-dashboard.json:ro
93+
networks:
94+
- semantic-network
95+
6796
networks:
6897
semantic-network:
6998
driver: bridge

‎website/docs/getting-started/observability.md‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Observability
2+
3+
Set up Prometheus + Grafana locally with the existing Docker Compose in this repo. The router already exposes Prometheus metrics and ships a ready-to-use Grafana dashboard, so you mainly need to run the services and ensure Prometheus points at the metrics endpoint.
4+
5+
## What’s included
6+
7+
- Router metrics server: `/metrics` on port `9190` (override with `--metrics-port`).
8+
- Classification API health check: `GET /health` on `8080` (`--api-port`).
9+
- Envoy (optional): admin on `19000`, Prometheus metrics at `/stats/prometheus`.
10+
- Docker Compose services: `semantic-router`, `envoy`, `prometheus`, `grafana` on the same `semantic-network`.
11+
- Grafana dashboard: `deploy/llm-router-dashboard.json` (auto-provisioned).
12+
13+
Code reference: `src/semantic-router/cmd/main.go` uses `promhttp` to expose `/metrics` (default `:9190`).
14+
15+
## Files to know
16+
17+
- Prometheus config: `config/prometheus.yaml`. Ensure the path matches the volume mount in `docker-compose.yml`.
18+
- Grafana provisioning:
19+
- Datasource: `config/grafana/datasource.yaml`
20+
- Dashboards: `config/grafana/dashboards.yaml`
21+
- Dashboard JSON: `deploy/llm-router-dashboard.json`
22+
23+
These files are already referenced by `docker-compose.yml` so you typically don’t need to edit them unless you’re changing targets or credentials.
24+
25+
## How it works (local)
26+
27+
- Prometheus runs in the same Docker network and scrapes `semantic-router:9190/metrics`. No host port needs to be published for metrics.
28+
- Grafana connects to Prometheus via the internal URL `http://prometheus:9090` and auto-loads the bundled dashboard.
29+
- Envoy (if enabled) can also be scraped by Prometheus at `envoy-proxy:19000/stats/prometheus`.
30+
31+
## Start and access
32+
33+
1) From the project root, start Compose (Prometheus and Grafana are included in the provided file).
34+
35+
```bash
36+
# try it out with mock-vllm
37+
CONFIG_FILE=/app/config/config.testing.yaml docker compose --profile testing up --build
38+
```
39+
40+
2) Open the UIs:
41+
- Prometheus: http://localhost:9090
42+
- Grafana: http://localhost:3000 (default admin/admin — change on first login)
43+
3) In Grafana, the “LLM Router” dashboard is pre-provisioned. If needed, import `deploy/llm-router-dashboard.json` manually.
44+
45+
## Minimal expectations
46+
47+
- Prometheus should list targets for:
48+
- `semantic-router:9190` (required)
49+
- `envoy-proxy:19000` (optional)
50+
- Grafana’s datasource should point to `http://prometheus:9090` inside the Docker network.
51+
52+
That’s it—run the stack, and you’ll have Prometheus scraping the router plus a prebuilt Grafana dashboard out of the box.

‎website/sidebars.js‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ const sidebars = {
5151
'getting-started/docker-quickstart',
5252
'getting-started/reasoning',
5353
'getting-started/configuration',
54+
'getting-started/observability',
5455
],
5556
},
5657
{

0 commit comments

Comments
 (0)