You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-inference-server-http.md
+16-16Lines changed: 16 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -192,7 +192,7 @@ For attach mode, take the following steps to use VS Code with the Python extensi
192
192
193
193
1. In the command window, search the logs from the inference server to locate the process ID of the `azmlinfsrv` process:
194
194
195
-
:::image type="content" source="./media/how-to-inference-server-http/debug-attach-pid.png" border="false" alt-text="Screenshot of a command window that shows inference server logs. In one log statement, the process ID of the azmlinfsrv command is highlighted.":::
195
+
:::image type="content" source="media/how-to-inference-server-http/debug-attach-pid.png" border="false" alt-text="Screenshot of a command window that shows inference server logs. In one log statement, the process ID of the azmlinfsrv command is highlighted." lightbox="media/how-to-inference-server-http/debug-attach-pid.png":::
196
196
197
197
Be sure to locate the ID of the `azmlinfsrv` process, not the `gunicorn` process.
198
198
@@ -232,7 +232,7 @@ The following procedure runs the inference server locally with [sample files](ht
232
232
233
233
During deployment, the `AZUREML_MODEL_DIR` variable is defined to store the path to the model folder. You specify that value in the `model_dir` parameter. When the scoring script runs, it retrieves the value from the `AZUREML_MODEL_DIR` variable.
234
234
235
-
In this case, use the current directory, `./`, as the `model_dir` value, because the scoring script specifies the subdirectory as `model/sklearn\_regression\_model.pkl`.
235
+
In this case, use the current directory, `./`, as the `model_dir` value, because the scoring script specifies the subdirectory as `model/sklearn_regression_model.pkl`.
@@ -266,12 +266,12 @@ The inference server accepts the following parameters:
266
266
267
267
| Parameter | Required | Default | Description |
268
268
| --- | --- | :---: | --- |
269
-
|`entry_script`| True | N/A | Identifies the relative or absolute path to the scoring script.|
270
-
|`model_dir`| False | N/A | Identifies the relative or absolute path to the directory that holds the model used for inferencing.|
271
-
|`port`| False | 5001 | Specifies the serving port of the inference server.|
272
-
|`worker_count`| False | 1 | Provides the number of worker threads to process concurrent requests.|
273
-
|`appinsights_instrumentation_key`| False | N/A | Provides the instrumentation key for the instance of Application Insights where the logs are published.|
274
-
|`access_control_allow_origins`| False | N/A | Turns on cross-origin resource sharing (CORS) for the specified origins, where multiple origins are separated by a comma (,), such as `microsoft.com, bing.com`.|
269
+
|`entry_script`| True | N/A | Identifies the relative or absolute path to the scoring script |
270
+
|`model_dir`| False | N/A | Identifies the relative or absolute path to the directory that holds the model used for inferencing |
271
+
|`port`| False | 5001 | Specifies the serving port of the inference server |
272
+
|`worker_count`| False | 1 | Provides the number of worker threads to process concurrent requests |
273
+
|`appinsights_instrumentation_key`| False | N/A | Provides the instrumentation key for the instance of Application Insights where the logs are published |
274
+
|`access_control_allow_origins`| False | N/A | Turns on cross-origin resource sharing (CORS) for the specified origins, where multiple origins are separated by a comma (,), such as `microsoft.com, bing.com`|
275
275
276
276
## Explore inference server request processing
277
277
@@ -290,7 +290,7 @@ The following steps demonstrate how the inference server, `azmlinfsrv`, handles
290
290
291
291
1. Your entry script receives the request. The entry script makes an inference call to the loaded model and returns a response.
292
292
293
-
:::image type="content" source="./media/how-to-inference-server-http/inference-server-architecture.png" border="false" alt-text="Diagram that shows how the inference server processes incoming requests." lightbox="./media/how-to-inference-server-http/inference-server-architecture.png":::
293
+
:::image type="content" source="./media/how-to-inference-server-http/inference-server-architecture.png" border="false" alt-text="Diagram that shows how the inference server starts and how a request flows to a Flask worker app and then to user code." lightbox="./media/how-to-inference-server-http/inference-server-architecture.png":::
294
294
295
295
## Explore inference server logs
296
296
@@ -334,7 +334,7 @@ Score: POST 127.0.0.1:<port>/score
334
334
<logs>
335
335
```
336
336
337
-
For example, when you run the inference server by taking the [end-to-end example](#use-an-end-to-end-example) steps, the log displays the following information:
337
+
For example, when you run the inference server by taking the [end-to-end example](#use-an-end-to-end-example) steps, the logs contain the following information:
338
338
339
339
```console
340
340
Azure ML Inferencing HTTP server v1.2.2
@@ -381,13 +381,13 @@ All logs from the inference server, except the launcher script, present data in
- `<UTC-time>`: The time when the entry is entered into the log.
387
-
- `<level>`: The first character of the [logging level](https://docs.python.org/3/library/logging.html#logging-levels) for the entry, such as `E` for ERROR, `I` for INFO, and so on.
388
-
- `<process-ID>`: The ID of the process associated with the entry.
389
-
- `<logger-name>`: The name of the resource associated with the log entry.
390
-
- `<message>`: The contents of the log message.
386
+
- `<UTC-time>`: The time when the entry is entered into the log
387
+
- `<level>`: The first character of the [logging level](https://docs.python.org/3/library/logging.html#logging-levels) for the entry, such as `E` for ERROR, `I` for INFO, and so on
388
+
- `<process-ID>`: The ID of the process associated with the entry
389
+
- `<logger-name>`: The name of the resource associated with the log entry
390
+
- `<message>`: The contents of the log message
391
391
392
392
There are six levels of logging in Python. Each level has an assigned numeric value according to its severity:
0 commit comments