Inference image and server updates

Author: shivanissambare

articles/machine-learning/concept-prebuilt-docker-images-inference.md

@@ -7,17 +7,16 @@ ms.service: machine-learning
 ms.subservice: core
 ms.author: ssambare
 author: shivanissambare
-ms.date: 10/21/2021
+ms.date: 07/14/2022
 ms.topic: conceptual
 ms.reviewer: larryfr
-ms.custom: deploy, docker, prebuilt, curated environments
+ms.custom: deploy, docker, prebuilt
 ---
 
 # Prebuilt Docker images for inference
 
 Prebuilt Docker container images for inference are used when deploying a model with Azure Machine Learning.  The images are prebuilt with popular machine learning frameworks and Python packages. You can also extend the packages to add other packages by using one of the following methods:
 
-* [Add Python packages](how-to-prebuilt-docker-images-inference-python-extensibility.md).
 * [Use prebuilt inference image as base for a new Dockerfile](how-to-extend-prebuilt-docker-image-inference.md). Using this method, you can install both **Python packages and apt packages**.
 
 ## Why should I use prebuilt images?
@@ -29,6 +28,9 @@ Prebuilt Docker container images for inference are used when deploying a model w
 
 ## List of prebuilt Docker images for inference 
 
+> [!IMPORTANT]
+> The list provided below includes only **currently supported** inference images by Azure Machine Learning.
+
 [!INCLUDE [list-of-inference-prebuilt-docker-images](../../includes/aml-inference-list-prebuilt-docker-images.md)]
 
 ## Next steps

articles/machine-learning/how-to-inference-server-http.md

@@ -10,7 +10,7 @@ ms.service: machine-learning
 ms.subservice: core
 ms.topic: how-to
 ms.custom: inference server, local development, local debugging, devplatv2
-ms.date: 05/14/2021
+ms.date: 07/14/2022
 ---
 
 # Azure Machine Learning inference HTTP server (preview)
@@ -135,15 +135,53 @@ There are two ways to use Visual Studio Code (VSCode) and [Python Extension](htt
 1. User starts the AzureML Inference Server in a command line and use VSCode + Python Extension to attach to the process.
 1. User sets up the `launch.json` in the VSCode and start the AzureML Inference Server within VSCode.
 
+**launch.json**
+```json
+{
+    "name": "Debug score.py",
+    "type": "python",
+    "request": "launch",
+    "module": "azureml_inference_server_http.amlserver",
+    "args": [
+        "--entry_script",
+        "score.py"
+    ]
+}
+```
+
 In both ways, user can set breakpoint and debug step by step.
 
 ## Frequently asked questions
 
-### Do I need to reload the server when changing the score script?
+### 1. I encountered the following error during server startup:
+
+```bash
+
+TypeError: register() takes 3 positional arguments but 4 were given
+
+  File "/var/azureml-server/aml_blueprint.py", line 251, in register
+
+    super(AMLBlueprint, self).register(app, options, first_registration)
+
+TypeError: register() takes 3 positional arguments but 4 were given
+
+```
+
+You have **Flask 2** installed in your python environment but are running a server (< 7.0.0) that does not support Flask 2. To resolve, please upgrade to the latest version of server.
+
+### 2. I encountered an ``ImportError`` or ``ModuleNotFoundError`` on modules ``opencensus``, ``jinja2``, ``MarkupSafe``, or ``click`` during startup like the following:
+
+```bash
+ImportError: cannot import name 'Markup' from 'jinja2'
+```
+
+Some older versions of the server do not have the correct boundaries for these dependencies. Please upgrade to the latest version of the server. We also recommend to not include these packages in their dependencies and rely on our server package to install the compatible versions of these dependencies.
+
+### 3. Do I need to reload the server when changing the score script?
 
 After changing your scoring script (`score.py`), stop the server with `ctrl + c`. Then restart it with `azmlinfsrv --entry_script score.py`.
 
-### Which OS is supported?
+### 4. Which OS is supported?
 
 The Azure Machine Learning inference server runs on Windows & Linux based operating systems.
 

articles/machine-learning/resource-curated-environments.md

@@ -29,7 +29,7 @@ This article lists the curated environments with latest framework versions in Az
 >[!IMPORTANT] 
 > To view more information about curated environment packages and versions, visit the Environments tab in the Azure Machine Learning [studio](./how-to-manage-environments-in-studio.md). 
 
-## Training curated environments
+## Curated environments
 
 ### PyTorch
 
@@ -89,9 +89,5 @@ Azure ML pipeline training workflows that use AutoML automatically selects a cur
 
 For more information on AutoML and Azure ML pipelines, see [use automated ML in an Azure Machine Learning pipeline in Python](how-to-use-automlstep-in-pipelines.md).
 
-## Inference curated environments and prebuilt docker images
-
-[!INCLUDE [list-of-inference-prebuilt-docker-images](../../includes/aml-inference-list-prebuilt-docker-images.md)]
-
 ## Support
 Version updates for supported environments, including the base images they reference, are released every two weeks to address vulnerabilities no older than 30 days. Based on usage, some environments may be deprecated (hidden from the product but usable) to support more common machine learning scenarios.

includes/aml-inference-list-prebuilt-docker-images.md

@@ -7,47 +7,49 @@ ms.service: machine-learning
 ms.author: ssambare
 ms.custom: "include file"
 ms.topic: "include"
-ms.date: 10/07/2021
+ms.date: 07/14/2022
 ---
 
 * All the docker images run as non-root user.
-* We recommend using `latest` tag for docker images. Prebuilt docker images for inference are published to Microsoft container registry (MCR), to query list of tags available, follow [instructions on their GitHub repository](https://github.com/microsoft/ContainerRegistry#browsing-mcr-content).
+* We recommend using `latest` tag for docker images. Prebuilt docker images for inference are published to Microsoft container registry (MCR), to query list of tags available, follow [instructions on the GitHub repository](https://github.com/microsoft/ContainerRegistry#browsing-mcr-content).
+
+### No framework
+
+Framework version | CPU/GPU | Pre-installed packages | MCR Path
+ --- | --- | --- | --- |
+NA | CPU | NA | `mcr.microsoft.com/azureml/minimal-ubuntu18.04-py37-cpu-inference:latest`
+NA | GPU | NA | `mcr.microsoft.com/azureml/minimal-ubuntu18.04-py37-cuda11.0.3-gpu-inference:latest`
 
 ### TensorFlow
 
-Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment
- --- | --- | --- | --- | --- |
- 1.15 | CPU | pandas==0.25.1 </br> numpy=1.20.1 | `mcr.microsoft.com/azureml/tensorflow-1.15-ubuntu18.04-py37-cpu-inference:latest`  | AzureML-tensorflow-1.15-ubuntu18.04-py37-cpu-inference | 
-2.4 | CPU | numpy>=1.16.0 </br> pandas~=1.1.x | `mcr.microsoft.com/azureml/tensorflow-2.4-ubuntu18.04-py37-cpu-inference:latest` | AzureML-tensorflow-2.4-ubuntu18.04-py37-cpu-inference |
-2.4 | GPU | numpy >= 1.16.0 </br> pandas~=1.1.x </br> CUDA==11.0.3 </br> CuDNN==8.0.5.39 | `mcr.microsoft.com/azureml/tensorflow-2.4-ubuntu18.04-py37-cuda11.0.3-gpu-inference:latest` | AzureML-tensorflow-2.4-ubuntu18.04-py37-cuda11.0.3-gpu-inference |
+Framework version | CPU/GPU | Pre-installed packages | MCR Path 
+ --- | --- | --- | --- |
+2.4 | CPU | numpy>=1.16.0 </br> pandas~=1.1.x | `mcr.microsoft.com/azureml/tensorflow-2.4-ubuntu18.04-py37-cpu-inference:latest` 
+2.4 | GPU | numpy >= 1.16.0 </br> pandas~=1.1.x </br> CUDA==11.0.3 </br> CuDNN==8.0.5.39 | `mcr.microsoft.com/azureml/tensorflow-2.4-ubuntu18.04-py37-cuda11.0.3-gpu-inference:latest`
+1.15 | CPU | pandas == 0.25.1 </br> numpy == 1.20.1 | `mcr.microsoft.com/azureml/tensorflow-1.15-ubuntu18.04-py37-cpu-inference:latest` 
 
 ### PyTorch
 
-Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment
- --- | --- | --- | --- | --- |
- 1.6 | CPU | numpy==1.20.1 </br> pandas==0.25.1 | `mcr.microsoft.com/azureml/pytorch-1.6-ubuntu18.04-py37-cpu-inference:latest` | AzureML-pytorch-1.6-ubuntu18.04-py37-cpu-inference |
-1.7 | CPU | numpy>=1.16.0 </br> pandas~=1.1.x | `mcr.microsoft.com/azureml/pytorch-1.7-ubuntu18.04-py37-cpu-inference:latest` | AzureML-pytorch-1.7-ubuntu18.04-py37-cpu-inference |
+Framework version | CPU/GPU | Pre-installed packages | MCR Path
+ --- | --- | --- | --- |
+1.10 | CPU | numpy >= 1.16.0 </br> pandas >= 1.1, < 1.2 | `mcr.microsoft.com/azureml/pytorch-1.10-ubuntu18.04-py37-cpu-inference` 
+1.9 | CPU | numpy >= 1.16.0 </br> pandas >= 1.1, < 1.2 | `mcr.microsoft.com/azureml/pytorch-1.9-ubuntu18.04-py37-cpu-inference:latest` 
+1.9 | GPU | | `mcr.microsoft.com/azureml/pytorch-1.9-ubuntu18.04-py37-cuda11.0.3-gpu-inference:latest` 
 
 ### SciKit-Learn
 
-Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment
- --- | --- | --- | --- | --- |
-0.24.1  | CPU | scikit-learn==0.24.1 </br> numpy>=1.16.0 </br> pandas~=1.1.x | `mcr.microsoft.com/azureml/sklearn-0.24.1-ubuntu18.04-py37-cpu-inference:latest` | AzureML-sklearn-0.24.1-ubuntu18.04-py37-cpu-inference |
+Framework version | CPU/GPU | Pre-installed packages | MCR Path
+ --- | --- | --- | --- | 
+0.24.1  | CPU | scikit-learn==0.24.1 </br> numpy>=1.16.0 </br> pandas~=1.1.x | `mcr.microsoft.com/azureml/sklearn-0.24.1-ubuntu18.04-py37-cpu-inference:latest`
 
-### ONNX Runtime
+### LightGBM
 
-Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment
- --- | --- | --- | --- | --- |
-1.6 | CPU | numpy>=1.16.0 </br> pandas~=1.1.x | `mcr.microsoft.com/azureml/onnxruntime-1.6-ubuntu18.04-py37-cpu-inference:latest` |AzureML-onnxruntime-1.6-ubuntu18.04-py37-cpu-inference |
+Framework version | CPU/GPU | Pre-installed packages | MCR Path
+ --- | --- | --- | --- |
+3.2 | CPU | scikit-learn==0.24.1 </br> numpy>=1.16.0, <1.20 </br> pandas>=1.1, <1.2 | `mcr.microsoft.com/azureml/lightgbm-3.2-ubuntu18.04-py37-cpu-inference:latest`
 
 ### XGBoost
 
-Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment
- --- | --- | --- | --- | --- |
-0.9 | CPU | scikit-learn==0.23.2 </br> numpy==1.20.1 </br> pandas==0.25.1 | `mcr.microsoft.com/azureml/xgboost-0.9-ubuntu18.04-py37-cpu-inference:latest` | AzureML-xgboost-0.9-ubuntu18.04-py37-cpu-inference | 
-
-### No framework
-
-Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment
- --- | --- | --- | --- | --- |
-NA | CPU | NA | `mcr.microsoft.com/azureml/minimal-ubuntu18.04-py37-cpu-inference:latest` | AzureML-minimal-ubuntu18.04-py37-cpu-inference  |
+Framework version | CPU/GPU | Pre-installed packages | MCR Path
+ --- | --- | --- | --- |
+0.9 | CPU | scikit-learn==0.23.2 </br> numpy==1.20.1 </br> pandas==0.25.1 | `mcr.microsoft.com/azureml/xgboost-0.9-ubuntu18.04-py37-cpu-inference:latest` |