Skip to content

Commit e2ff170

Browse files
xanwalxanwal
committed
Commit code from approved azureml-examples PR
1 parent 1e453cf commit e2ff170

File tree

1 file changed

+241
-0
lines changed

1 file changed

+241
-0
lines changed

‎articles/machine-learning/how-to-debug-managed-online-endpoints-visual-studio-code.md

Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,8 @@ The following table provides an overview of scenarios to help you choose what wo
4242

4343
## Prerequisites
4444

45+
# [Portal](#tab/azure-portal)
46+
4547
This guide assumes you have the following items installed locally on your PC.
4648

4749
- [Docker](https://docs.docker.com/engine/install/)
@@ -74,8 +76,48 @@ az account set --subscription <subscription>
7476
az configure --defaults workspace=<workspace> group=<resource-group> location=<location>
7577
```
7678

79+
# [Python](#tab/python)
80+
[!INCLUDE [sdk v2](../../includes/machine-learning-sdk-v2.md)]
81+
82+
This guide assumes you have the following items installed locally on your PC.
83+
84+
- [Docker](https://docs.docker.com/engine/install/)
85+
- [VS Code](https://code.visualstudio.com/#alt-downloads)
86+
- [Azure CLI](/cli/azure/install-azure-cli)
87+
- [Azure CLI `ml` extension (v2)](how-to-configure-cli.md)
88+
- [Azure ML Python SDK (v2)](https://aka.ms/sdk-v2-install)
89+
90+
For more information, see the guide on [how to prepare your system to deploy managed online endpoints](how-to-deploy-managed-online-endpoints.md#prepare-your-system).
91+
92+
The examples in this article are based on code samples contained in the [azureml-examples](https://github.com/azure/azureml-examples) repository. To run the commands locally without having to copy/paste YAML and other files, clone the repo and then change directories to the `cli` directory in the repo:
93+
94+
```azurecli
95+
git clone https://github.com/Azure/azureml-examples --depth 1
96+
cd azureml-examples
97+
cd cli
98+
```
99+
100+
If you haven't already set the defaults for the Azure CLI, save your default settings. To avoid passing in the values for your subscription, workspace, and resource group multiple times, use the following commands. Replace the following parameters with values for your specific configuration:
101+
102+
* Replace `<subscription>` with your Azure subscription ID.
103+
* Replace `<workspace>` with your Azure Machine Learning workspace name.
104+
* Replace `<resource-group>` with the Azure resource group that contains your workspace.
105+
* Replace `<location>` with the Azure region that contains your workspace.
106+
107+
> [!TIP]
108+
> You can see what your current defaults are by using the `az configure -l` command.
109+
110+
```azurecli
111+
az account set --subscription <subscription>
112+
az configure --defaults workspace=<workspace> group=<resource-group> location=<location>
113+
```
114+
115+
---
116+
77117
## Launch development container
78118

119+
# [Portal](#tab/azure-portal)
120+
79121
Azure Machine Learning local endpoints use Docker and VS Code development containers (dev container) to build and configure a local debugging environment. With dev containers, you can take advantage of VS Code features from inside a Docker container. For more information on dev containers, see [Create a development container](https://code.visualstudio.com/docs/remote/create-dev-container).
80122

81123
To debug online endpoints locally in VS Code, use the `--vscode-debug` flag when creating or updating and Azure Machine Learning online deployment. The following command uses a deployment example from the examples repo:
@@ -104,6 +146,102 @@ You'll use a few VS Code extensions to debug your deployments in the dev contain
104146
> [!IMPORTANT]
105147
> Before starting your debug session, make sure that the VS Code extensions have finished installing in your dev container.
106148
149+
150+
# [Python](#tab/python)
151+
152+
Azure Machine Learning local endpoints use Docker and VS Code development containers (dev container) to build and configure a local debugging environment. With dev containers, you can take advantage of VS Code features from inside a Docker container. For more information on dev containers, see [Create a development container](https://code.visualstudio.com/docs/remote/create-dev-container).
153+
154+
Import the required modules:
155+
156+
```python
157+
from azure.ai.ml import MLClient
158+
from azure.ai.ml.entities import (
159+
ManagedOnlineEndpoint,
160+
ManagedOnlineDeployment,
161+
Model,
162+
CodeConfiguration,
163+
Environment,
164+
)
165+
from azure.identity import DefaultAzureCredential, AzureCliCredential
166+
```
167+
168+
Set up variables for the workspace and endpoint:
169+
170+
```python
171+
subscription_id = "<SUBSCRIPTION_ID>"
172+
resource_group = "<RESOURCE_GROUP>"
173+
workspace_name = "<AML_WORKSPACE_NAME>"
174+
175+
endpoint_name = "<ENDPOINT_NAME>"
176+
```
177+
178+
Get a handle to the workspace:
179+
180+
```python
181+
182+
credential = AzureCliCredential()
183+
ml_client = MLClient(
184+
credential,
185+
subscription_id=subscription_id,
186+
resource_group_name=resource_group,
187+
workspace_name=workspace_name,
188+
)
189+
190+
```
191+
192+
To debug online endpoints locally in VS Code, set the `vscode-debug` and `local` flags when creating or updating an Azure Machine Learning online deployment. The following code mirrors a deployment example from the examples repo:
193+
194+
```python
195+
196+
deployment = ManagedOnlineDeployment(
197+
name="blue",
198+
endpoint_name=endpoint_name,
199+
model=Model(path="../model-1/model"),
200+
code_configuration=CodeConfiguration(
201+
code="../model-1/onlinescoring", scoring_script="score.py"
202+
),
203+
environment=Environment(
204+
conda_file="../model-1/environment/conda.yml",
205+
image="mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20210727.v1",
206+
),
207+
instance_type="Standard_DS2_v2",
208+
instance_count=1,
209+
)
210+
211+
deployment = ml_client.online_deployments.begin_create_or_update(
212+
deployment,
213+
local=True,
214+
vscode_debug=True,
215+
)
216+
217+
```
218+
219+
> [!IMPORTANT]
220+
> On Windows Subsystem for Linux (WSL), you'll need to update your PATH environment variable to include the path to the VS Code executable or use WSL interop. For more information, see [Windows interoperability with Linux](/windows/wsl/interop).
221+
222+
A Docker image is built locally. Any environment configuration or model file errors are surfaced at this stage of the process.
223+
224+
> [!NOTE]
225+
> The first time you launch a new or updated dev container it can take several minutes.
226+
227+
Once the image successfully builds, your dev container opens in a VS Code window.
228+
229+
You'll use a few VS Code extensions to debug your deployments in the dev container. Azure Machine Learning automatically installs these extensions in your dev container.
230+
231+
- Inference Debug
232+
- [Pylance](https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance)
233+
- [Jupyter](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter)
234+
- [Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python)
235+
236+
> [!IMPORTANT]
237+
> Before starting your debug session, make sure that the VS Code extensions have finished installing in your dev container.
238+
239+
---
240+
241+
242+
243+
---
244+
107245
## Start debug session
108246

109247
Once your environment is set up, use the VS Code debugger to test and debug your deployment locally.
@@ -136,6 +274,8 @@ For more information on the VS Code debugger, see [Debugging in VS Code](https:/
136274

137275
## Debug your endpoint
138276

277+
# [Portal](#tab/azure-portal)
278+
139279
Now that your application is running in the debugger, try making a prediction to debug your scoring script.
140280

141281
Use the `ml` extension `invoke` command to make a request to your local endpoint.
@@ -180,8 +320,68 @@ In this case, `<REQUEST-FILE>` is a JSON file that contains input data samples f
180320
181321
At this point, any breakpoints in your `run` function are caught. Use the debug actions to step through your code. For more information on debug actions, see the [debug actions guide](https://code.visualstudio.com/Docs/editor/debugging#_debug-actions).
182322
323+
324+
# [Python](#tab/python)
325+
326+
Now that your application is running in the debugger, try making a prediction to debug your scoring script.
327+
328+
Use the`invoke` method on your `MLClient` object to make a request to your local endpoint.
329+
330+
```python
331+
332+
endpoint = ml_client.online_endpoints.get(name=endpoint_name, local=True)
333+
334+
request_file_path = "../model-1/sample-request.json"
335+
336+
endpoint.invoke(endpoint_name, request_file_path, local=True)
337+
338+
```
339+
340+
In this case, `<REQUEST-FILE>` is a JSON file that contains input data samples for the model to make predictions on similar to the following JSON:
341+
342+
```json
343+
{"data": [
344+
[1,2,3,4,5,6,7,8,9,10],
345+
[10,9,8,7,6,5,4,3,2,1]
346+
]}
347+
```
348+
349+
> [!TIP]
350+
> The scoring URI is the address where your endpoint listens for requests. The `as_dict` method of endpoint objects returns information similar to `show` in the Azure CLI. The endpoint object can be obtained through `.get`.
351+
>
352+
> ```python
353+
> endpoint = ml_client.online_endpoints.get(endpoint_name, local=True)
354+
endpoint.as_dict()
355+
> ```
356+
>
357+
> The output should look similar to the following:
358+
>
359+
> ```json
360+
> {
361+
> "auth_mode": "aml_token",
362+
> "location": "local",
363+
> "name": "my-new-endpoint",
364+
> "properties": {},
365+
> "provisioning_state": "Succeeded",
366+
> "scoring_uri": "http://localhost:5001/score",
367+
> "tags": {},
368+
> "traffic": {},
369+
> "type": "online"
370+
>}
371+
>```
372+
>
373+
>The scoring URI can be found in the `scoring_uri` key.
374+
375+
At this point, any breakpoints in your `run` function are caught. Use the debug actions to step through your code. For more information on debug actions, see the [debug actions guide](https://code.visualstudio.com/Docs/editor/debugging#_debug-actions).
376+
377+
378+
---
379+
380+
183381
## Edit your endpoint
184382
383+
# [Portal](#tab/azure-portal)
384+
185385
As you debug and troubleshoot your application, there are scenarios where you need to update your scoring script and configurations.
186386
187387
To apply changes to your code:
@@ -200,6 +400,47 @@ az ml online-deployment update --file <DEPLOYMENT-YAML-SPECIFICATION-FILE> --loc
200400
201401
Once the updated image is built and your development container launches, use the VS Code debugger to test and troubleshoot your updated endpoint.
202402
403+
# [Python](#tab/python)
404+
405+
As you debug and troubleshoot your application, there are scenarios where you need to update your scoring script and configurations.
406+
407+
To apply changes to your code:
408+
409+
1. Update your code
410+
1. Restart your debug session using the `Developer: Reload Window` command in the command palette. For more information, see the [command palette documentation](https://code.visualstudio.com/docs/getstarted/userinterface#_command-palette).
411+
412+
> [!NOTE]
413+
> Since the directory containing your code and endpoint assets is mounted onto the dev container, any changes you make in the dev container are synced with your local file system.
414+
415+
For more extensive changes involving updates to your environment and endpoint configuration, use the `ml` extension `update` command. Doing so will trigger a full image rebuild with your changes.
416+
417+
```python
418+
419+
new_deployment = ManagedOnlineDeployment(
420+
name="green",
421+
endpoint_name=endpoint_name,
422+
model=Model(path="../model-2/model"),
423+
code_configuration=CodeConfiguration(
424+
code="../model-2/onlinescoring", scoring_script="score.py"
425+
),
426+
environment=Environment(
427+
conda_file="../model-1/environment/conda.yml",
428+
image="mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20210727.v1",
429+
),
430+
instance_type="Standard_DS2_v2",
431+
instance_count=2,
432+
)
433+
434+
ml_client.online_deployments.update(new_deployment, local=True, vscode_debug=True)
435+
436+
```
437+
438+
Once the updated image is built and your development container launches, use the VS Code debugger to test and troubleshoot your updated endpoint.
439+
440+
441+
442+
---
443+
203444
## Next steps
204445
205446
- [Deploy and score a machine learning model by using a managed online endpoint (preview)](how-to-deploy-managed-online-endpoints.md)

0 commit comments

Comments
 (0)