You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
### Deploy the InferencePool and Endpoint Picker Extension
84
+
85
+
Install an InferencePool named `vllm-llama3-8b-instruct` that selects from endpoints with label app: vllm-llama3-8b-instruct and listening on port 8000, you can run the following command:
86
+
87
+
```bash
88
+
export GATEWAY_PROVIDER=none # See [README](https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/config/charts/inferencepool/README.md#configuration) for valid configurations
6. Confirm that the HealthCheckPolicy status conditions include `Attached=True`:
139
+
140
+
```bash
141
+
kubectl get healthcheckpolicy health-check-policy -o yaml
142
+
```
116
143
117
144
=== "Istio"
118
145
@@ -267,22 +294,6 @@ A cluster with:
267
294
kubectl get httproute llm-route -o yaml
268
295
```
269
296
270
-
271
-
### Deploy the InferencePool and Endpoint Picker Extension
272
-
273
-
Install an InferencePool named `vllm-llama3-8b-instruct` that selects from endpoints with label app: vllm-llama3-8b-instruct and listening on port 8000, you can run the following command:
274
-
275
-
```bash
276
-
export GATEWAY_PROVIDER=none # See [README](https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/config/charts/inferencepool/README.md#configuration) for valid configurations
0 commit comments