| 
 | 1 | +# Docker Compose Quick Start  | 
 | 2 | + | 
 | 3 | +Run Semantic Router + Envoy locally using Docker Compose v2.  | 
 | 4 | + | 
 | 5 | +## Prerequisites  | 
 | 6 | + | 
 | 7 | +- Docker Engine and Docker Compose v2 (use the `docker compose` command, not the legacy `docker-compose`)  | 
 | 8 | +- Ensure ports 8801, 50051, 19000 are free  | 
 | 9 | + | 
 | 10 | +## Install and Run with Docker Compose v2  | 
 | 11 | + | 
 | 12 | +1) Clone the repo and move into it (from your workspace root):  | 
 | 13 | + | 
 | 14 | +```bash  | 
 | 15 | +git clone https://github.com/vllm-project/semantic-router.git  | 
 | 16 | +cd semantic-router  | 
 | 17 | +```  | 
 | 18 | + | 
 | 19 | +2) Download required models (classification models):  | 
 | 20 | + | 
 | 21 | +```bash  | 
 | 22 | +make download-models  | 
 | 23 | +```  | 
 | 24 | + | 
 | 25 | +This downloads the classification models used by the router:  | 
 | 26 | + | 
 | 27 | +- Category classifier (ModernBERT-base)  | 
 | 28 | +- PII classifier (ModernBERT-base)  | 
 | 29 | +- Jailbreak classifier (ModernBERT-base)  | 
 | 30 | + | 
 | 31 | +Note: The BERT similarity model defaults to a remote Hugging Face model. See Troubleshooting for offline/local usage.  | 
 | 32 | + | 
 | 33 | +3) Start the services with Docker Compose v2:  | 
 | 34 | + | 
 | 35 | +```bash  | 
 | 36 | +# Start core services (semantic-router + envoy)  | 
 | 37 | +docker compose up --build  | 
 | 38 | + | 
 | 39 | +# Or run in background (recommended)  | 
 | 40 | +docker compose up --build -d  | 
 | 41 | + | 
 | 42 | +# With testing profile (includes mock vLLM)  | 
 | 43 | +docker compose --profile testing up --build  | 
 | 44 | +```  | 
 | 45 | + | 
 | 46 | +4) Verify  | 
 | 47 | + | 
 | 48 | +- Semantic Router (gRPC): localhost:50051  | 
 | 49 | +- Envoy Proxy: http://localhost:8801  | 
 | 50 | +- Envoy Admin: http://localhost:19000  | 
 | 51 | + | 
 | 52 | +## Common Operations  | 
 | 53 | + | 
 | 54 | +```bash  | 
 | 55 | +# View service status  | 
 | 56 | +docker compose ps  | 
 | 57 | + | 
 | 58 | +# Follow logs for the router service  | 
 | 59 | +docker compose logs -f semantic-router  | 
 | 60 | + | 
 | 61 | +# Exec into the router container  | 
 | 62 | +docker compose exec semantic-router bash  | 
 | 63 | + | 
 | 64 | +# Stop and clean up containers  | 
 | 65 | +docker compose down  | 
 | 66 | +```  | 
 | 67 | + | 
 | 68 | +## Troubleshooting  | 
 | 69 | + | 
 | 70 | +### 1) Router exits immediately with a Hugging Face DNS/download error  | 
 | 71 | + | 
 | 72 | +Symptoms (from `docker compose logs -f semantic-router`):  | 
 | 73 | + | 
 | 74 | +```  | 
 | 75 | +Failed to initialize BERT: request error: https://huggingface.co/... Dns Failed: resolve dns name 'huggingface.co:443'  | 
 | 76 | +```  | 
 | 77 | + | 
 | 78 | +Why: `bert_model.model_id` in `config/config.yaml` points to a remote model (`sentence-transformers/all-MiniLM-L12-v2`). If the container cannot resolve or reach the internet, startup fails.  | 
 | 79 | + | 
 | 80 | +Fix options:  | 
 | 81 | + | 
 | 82 | +- Allow network access in the container (online):  | 
 | 83 | +  - Ensure your host can resolve DNS, or add DNS servers to the `semantic-router` service in `docker-compose.yml`:  | 
 | 84 | + | 
 | 85 | +      ```yaml  | 
 | 86 | +      services:  | 
 | 87 | +         semantic-router:  | 
 | 88 | +            # ...  | 
 | 89 | +            dns:  | 
 | 90 | +               - 1.1.1.1  | 
 | 91 | +               - 8.8.8.8  | 
 | 92 | +      ```  | 
 | 93 | +        | 
 | 94 | +  - If behind a proxy, set `http_proxy/https_proxy/no_proxy` env vars for the service.  | 
 | 95 | + | 
 | 96 | +- Use a local copy of the model (offline):  | 
 | 97 | +   1. Download `sentence-transformers/all-MiniLM-L12-v2` to `./models/sentence-transformers/all-MiniLM-L12-v2/` on the host.  | 
 | 98 | +   2. Update `config/config.yaml` to use the local path (mounted into the container at `/app/models`):  | 
 | 99 | + | 
 | 100 | +       ```yaml  | 
 | 101 | +       bert_model:  | 
 | 102 | +          model_id: "models/sentence-transformers/all-MiniLM-L12-v2"  | 
 | 103 | +          threshold: 0.6  | 
 | 104 | +          use_cpu: true  | 
 | 105 | +       ```  | 
 | 106 | + | 
 | 107 | +   3. Recreate services: `docker compose up -d --build`  | 
 | 108 | + | 
 | 109 | +### 2) Port already in use  | 
 | 110 | + | 
 | 111 | +Make sure 8801, 50051, 19000 are not bound by other processes. Adjust ports in `docker-compose.yml` if needed.  | 
0 commit comments