Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 0 additions & 55 deletions deploy/docker-compose.tracing.yaml

This file was deleted.

42 changes: 42 additions & 0 deletions deploy/docker-compose/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Main Runtime Compose Stack

This directory contains the primary `docker-compose.yml` used to run the semantic-router stack (router + envoy + optional mock-vllm + observability).

## Path Layout
Because this file lives under `deploy/docker-compose/`, all relative paths to repository resources go two levels up (../../) back to repo root.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deploy/docker-compose/ maybe -> deploy/docker-compose ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do u mean moving the docker-compose.yml out of the dir? I just wanted to keep 3 deployments parallel.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no. What I mean is, can we remove the '/' at the end? It seems a bit counterintuitive,
but it's just a suggestion. Don't pay too much attention to it.


Example mappings:

- `../../config` -> mounts to `/app/config` inside containers
- `../../models` -> shared model files
- `../../tools/observability/...` -> Prometheus / Grafana provisioning assets

## Profiles

- `testing` : enables `mock-vllm` and `llm-katan`
- `llm-katan` : only `llm-katan`

## Common Commands

```bash
# Bring up core stack
docker compose -f deploy/docker-compose/docker-compose.yml up --build

# With testing profile (adds mock-vllm & llm-katan)
docker compose -f deploy/docker-compose/docker-compose.yml --profile testing up --build

# Tear down
docker compose -f deploy/docker-compose/docker-compose.yml down
```

## Overrides
You can place a `docker-compose.override.yml` at repo root and combine:

```bash
docker compose -f deploy/docker-compose/docker-compose.yml -f docker-compose.override.yml up -d
```

## Related Stacks

- Local observability only: `tools/observability/docker-compose.obs.yml`
- Tracing stack: `tools/tracing/docker-compose.tracing.yaml`
20 changes: 10 additions & 10 deletions docker-compose.yml → deploy/docker-compose/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ services:
# Semantic Router External Processor Service
semantic-router:
build:
context: .
context: ../../
dockerfile: Dockerfile.extproc
container_name: semantic-router
ports:
- "50051:50051"
volumes:
- ./config:/app/config:ro
- ./models:/app/models:ro
- ../../config:/app/config:ro
- ../../models:/app/models:ro
environment:
- LD_LIBRARY_PATH=/app/lib
- CONFIG_FILE=${CONFIG_FILE:-/app/config/config.yaml}
Expand All @@ -31,7 +31,7 @@ services:
- "8801:8801" # Main proxy port
- "19000:19000" # Admin interface
volumes:
- ./config/envoy-docker.yaml:/etc/envoy/envoy.yaml:ro
- ../../config/envoy-docker.yaml:/etc/envoy/envoy.yaml:ro
command: ["/usr/local/bin/envoy", "-c", "/etc/envoy/envoy.yaml", "--component-log-level", "ext_proc:trace,router:trace,http:trace"]
depends_on:
semantic-router:
Expand All @@ -48,7 +48,7 @@ services:
# Mock vLLM service for testing profile
mock-vllm:
build:
context: ./tools/mock-vllm
context: ../../tools/mock-vllm
dockerfile: Dockerfile
container_name: mock-vllm
profiles: ["testing"]
Expand All @@ -69,7 +69,7 @@ services:
image: prom/prometheus:v2.53.0
container_name: prometheus
volumes:
- ./tools/observability/prometheus.yaml:/etc/prometheus/prometheus.yaml:ro
- ../../tools/observability/prometheus.yaml:/etc/prometheus/prometheus.yaml:ro
- prometheus-data:/prometheus
command:
- --config.file=/etc/prometheus/prometheus.yaml
Expand All @@ -91,9 +91,9 @@ services:
ports:
- "3000:3000"
volumes:
- ./tools/observability/grafana-datasource.yaml:/etc/grafana/provisioning/datasources/datasource.yaml:ro
- ./tools/observability/grafana-dashboard.yaml:/etc/grafana/provisioning/dashboards/dashboard.yaml:ro
- ./tools/observability/llm-router-dashboard.json:/etc/grafana/provisioning/dashboards/llm-router-dashboard.json:ro
- ../../tools/observability/grafana-datasource.yaml:/etc/grafana/provisioning/datasources/datasource.yaml:ro
- ../../tools/observability/grafana-dashboard.yaml:/etc/grafana/provisioning/dashboards/dashboard.yaml:ro
- ../../tools/observability/llm-router-dashboard.json:/etc/grafana/provisioning/dashboards/llm-router-dashboard.json:ro
- grafana-data:/var/lib/grafana
networks:
- semantic-network
Expand All @@ -103,7 +103,7 @@ services:
# LLM Katan service for testing
llm-katan:
build:
context: ./e2e-tests/llm-katan
context: ../../e2e-tests/llm-katan
dockerfile: Dockerfile
container_name: llm-katan
profiles: ["testing", "llm-katan"]
Expand Down
Loading
Loading