You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: website/docs/installation/installation.md
+15-10Lines changed: 15 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,14 +8,16 @@ This guide will help you set up and install the Semantic Router on your system.
8
8
9
9
## System Requirements
10
10
11
-
**Note**: No GPU required - the router runs efficiently on CPU using optimized BERT models.
11
+
:::note
12
+
No GPU required - the router runs efficiently on CPU using optimized BERT models.
13
+
:::
12
14
13
-
### Software Dependencies
15
+
Semantic Router depends on the following software:
14
16
15
-
-**Go**: Version 1.24.1 or higher (matches the module requirements)
16
-
-**Rust**: Version 1.90.0 or higher (for Candle bindings)
17
-
-**Python**: Version 3.8 or higher (for model downloads)
18
-
-**HuggingFace CLI**: For model downloads (`pip install huggingface_hub`)
17
+
-**Go**: V1.24.1 or higher (matches the module requirements)
18
+
-**Rust**: V1.90.0 or higher (for Candle bindings)
19
+
-**Python**: V3.8 or higher (for model downloads)
20
+
-**HuggingFace CLI**: Required for fetching models (`pip install huggingface_hub`)
19
21
20
22
## Local Installation
21
23
@@ -94,7 +96,9 @@ This downloads the CPU-optimized BERT models for:
94
96
- PII detection
95
97
- Jailbreak detection
96
98
97
-
> **Tip:**`make test` invokes `make download-models` automatically, so you only need to run this step manually the first time or when refreshing the cache.
99
+
:::tip
100
+
`make test` invokes `make download-models` automatically, so you only need to run this step manually the first time or when refreshing the cache.
101
+
:::
98
102
99
103
### 5. Configure Backend Endpoints
100
104
@@ -118,8 +122,7 @@ model_config:
118
122
preferred_endpoints: ["your-endpoint"]
119
123
```
120
124
121
-
**⚠️ Important: Address Format Requirements**
122
-
125
+
:::note[**Important: Address Format Requirements**]
123
126
The `address` field **must** contain a valid IP address (IPv4 or IPv6). Domain names are not supported.
124
127
125
128
**✅ Correct formats:**
@@ -134,8 +137,9 @@ The `address` field **must** contain a valid IP address (IPv4 or IPv6). Domain n
134
137
- `"http://127.0.0.1"`→ Remove protocol prefix
135
138
- `"127.0.0.1:8080"`→ Use separate `port` field
136
139
137
-
**⚠️ Important: Model Name Consistency**
140
+
:::
138
141
142
+
:::note[**Important: Model Name Consistency**]
139
143
The model name in your configuration **must exactly match** the `--served-model-name` parameter used when starting your vLLM server:
140
144
141
145
```bash
@@ -154,6 +158,7 @@ model_config:
154
158
If these names don't match, the router won't be able to route requests to your model.
155
159
156
160
The default configuration includes example endpoints that you should update for your setup.
0 commit comments