Skip to content

Commit 1f492ef

Browse files
authored
explain routes for BYOC
1 parent d7ae187 commit 1f492ef

File tree

1 file changed

+7
-3
lines changed

1 file changed

+7
-3
lines changed

articles/machine-learning/how-to-deploy-custom-container.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -230,11 +230,15 @@ There are a few important concepts to notice in this YAML/Python parameter:
230230

231231
#### Readiness route vs. liveness route
232232

233-
An HTTP server defines paths for both _liveness_ and _readiness_. A liveness route is used to check whether the server is running. A readiness route is used to check whether the server is ready to do work. In machine learning inference, a server could respond 200 OK to a liveness request before loading a model. The server could respond 200 OK to a readiness request only after the model is loaded into memory.
233+
The API server you choose may provide a way to check the status of the server. There are two types of the route that you can specify: _liveness_ and _readiness_. A liveness route is used to check whether the server is running. A readiness route is used to check whether the server is ready to do work. In the context of machine learning inferencing, a server could respond 200 OK to a liveness request before loading a model, and the server could respond 200 OK to a readiness request only after the model is loaded into the memory.
234234

235-
For more information about liveness and readiness probes, see the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/).
235+
For more information about liveness and readiness probes in general, see the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/).
236236

237-
Notice that this deployment uses the same path for both liveness and readiness, since TF Serving only defines a liveness route.
237+
The liveness and readiness routes will be determined by the API server of your choice. Note that the example deployment in this article uses the same path for both liveness and readiness, since TFServing only defines a liveness route. Refer to other examples for different patterns to define the routes.
238+
239+
#### Scoring route
240+
241+
The API server you choose would provide a way to receive the payload to work on. In the context of machine learning inference, a server would receive the input data via a specific route. Identify this route for your API server and specify it when you define the deployment to create. Successful creation of the deployment will update the scoring_uri parameter of the endpoint as well, which you can verify with `az ml online-endpoint show -n <name> --query scoring_uri`.
238242

239243
#### Locating the mounted model
240244

0 commit comments

Comments
 (0)