Deploy: docker compose start failed

[yuluo-yx]

semantic-router            | {"level":"info","ts":"2025-10-15T09:44:19.822447994Z","caller":"observability/logging.go:141","msg":"Loaded jailbreak mapping with 2 jailbreak types"}
semantic-router            | {"level":"info","ts":"2025-10-15T09:44:19.822471161Z","caller":"candle-binding/semantic-router.go:276","msg":"Initializing BERT similarity model: sentence-transformers/all-MiniLM-L12-v2"}
openwebui                  | 2025-10-15 09:44:36.329 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - 172.28.0.1:37906 - "GET /_app/version.json HTTP/1.1" 200
semantic-router            | Failed to initialize BERT: request error: https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2/resolve/main/config.json: Connection Failed: Connect error: connection timed out
semantic-router            | {"level":"fatal","ts":"2025-10-15T09:44:49.855845538Z","caller":"observability/logging.go:145","msg":"Failed to create ExtProc server: failed to initialize BERT model: failed to initialize BERT similarity model","stacktrace":"github.com/vllm-project/semantic-router/src/semantic-router/pkg/observability.Fatalf\n\t/app/src/semantic-router/pkg/observability/logging.go:145\nmain.main\n\t/app/src/semantic-router/cmd/main.go:109\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:283"}
semantic-router exited with code 1
Gracefully stopping... (press Ctrl+C again to force)
dependency failed to start: container semantic-router exited (1)

Failed to initialize BERT: request error: https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2/resolve/main/config.json: Connection Failed: Connect error: connection timed out

It seems to be due to network problems. 👀

[yuluo-yx]

I will try to add - HF_ENDPOINT=https://hf-mirror.com in sr docker compose envirenment, but no have any help.

@JaredforReal What do you recommend? thx

[JaredforReal]

I will try to add - HF_ENDPOINT=https://hf-mirror.com in sr docker compose envirenment, but no have any help.

This one should help, did u try make docker-compose-up REBUILD=1 after adding HF_ENDPOINT?

[yuluo-yx]

I will try to add - HF_ENDPOINT=https://hf-mirror.com in sr docker compose envirenment, but no have any help.

This one should help, did u try make docker-compose-up REBUILD=1 after adding HF_ENDPOINT?

Nothing seems to help, I'll try again after #450 is merged. thx

[yuluo-yx]

 ✔ dashboard                                     Built                                                                                                                                                                                                  0.0s
 ✔ Network semantic-router_semantic-network      Created                                                                                                                                                                                                0.1s
 ✔ Volume "semantic-router_openwebui-data"       Created                                                                                                                                                                                                0.0s
 ✔ Volume "semantic-router_openwebui-pipelines"  Created                                                                                                                                                                                                0.0s
 ✔ Volume "semantic-router_hf-cache"             Created                                                                                                                                                                                                0.0s
 ✔ Volume "semantic-router_prometheus-data"      Created                                                                                                                                                                                                0.0s
 ✔ Volume "semantic-router_grafana-data"         Created                                                                                                                                                                                                0.0s
 ✔ Container llm-katan                           Started                                                                                                                                                                                                7.1s
 ✔ Container pipelines                           Started                                                                                                                                                                                                7.1s
 ✔ Container openwebui                           Started                                                                                                                                                                                                7.1s
 ✔ Container prometheus                          Started                                                                                                                                                                                                7.1s
 ✘ Container semantic-router                     Error                                                                                                                                                                                                 37.6s
 ✔ Container envoy-proxy                         Created                                                                                                                                                                                                0.1s
 ✔ Container grafana                             Started                                                                                                                                                                                                7.2s
 ✔ Container semantic-router-dashboard           Created                                                                                                                                                                                                0.1s
dependency failed to start: container semantic-router exited (1)
make[1]: *** [docker-compose-up] Error 1
make: *** [_run] Error 2

[entrypoint] Starting semantic-router with config: /app/config/config.yaml
[entrypoint] Additional args:
{"level":"info","ts":"2025-10-17T09:57:30.656976876Z","caller":"observability/logging.go:141","msg":"Starting metrics server on :9190"}
{"level":"info","ts":"2025-10-17T09:57:30.673906828Z","caller":"observability/logging.go:141","msg":"Loaded category mapping with 14 categories"}
{"level":"info","ts":"2025-10-17T09:57:30.681534483Z","caller":"observability/logging.go:141","msg":"Loaded PII mapping with 35 PII types"}
{"level":"info","ts":"2025-10-17T09:57:30.687673416Z","caller":"observability/logging.go:141","msg":"Loaded jailbreak mapping with 2 jailbreak types"}
{"level":"info","ts":"2025-10-17T09:57:30.687794292Z","caller":"candle-binding/semantic-router.go:276","msg":"Initializing BERT similarity model: sentence-transformers/all-MiniLM-L12-v2"}
Failed to initialize BERT: request error: https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2/resolve/main/config.json: Connection Failed: Connect error: connection timed out
{"level":"fatal","ts":"2025-10-17T09:58:00.754362213Z","caller":"observability/logging.go:145","msg":"Failed to create ExtProc server: failed to initialize BERT model: failed to initialize BERT similarity model","stacktrace":"github.com/vllm-project/semantic-router/src/semantic-router/pkg/observability.Fatalf\n\t/app/src/semantic-router/pkg/observability/logging.go:145\nmain.main\n\t/app/src/semantic-router/cmd/main.go:109\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:283"}

@JaredforReal The latest code execution make command still fails. 🤔

[JaredforReal]

Did u run make download-models first？ We are using pre-downloaded models/ in deployment

[yuluo-yx]

Did u run make download-models first？ We are using pre-downloaded models/ in deployment

yep, I run it.

[JaredforReal]

@yuluo-yx are u in VSR Slack?

[yuluo-yx]

@yuluo-yx are u in VSR Slack?

sorry, Failed to notice message, yep, I am.

Perhaps we could add each other on WeChat? 🤣