You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/batch-inference/how-to-access-data-batch-endpoints-jobs.md
+27-18Lines changed: 27 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,7 +23,7 @@ Batch endpoints can be used to perform batch scoring on large amounts of data. S
23
23
24
24
## Supported data inputs
25
25
26
-
Batch endpoints support reading files located in tje following storage options:
26
+
Batch endpoints support reading files located in the following storage options:
27
27
28
28
* Azure Machine Learning Data Stores. The following stores are supported:
29
29
* Azure Blob Storage
@@ -54,8 +54,11 @@ Data from Azure Machine Learning registered data stores can be directly referenc
54
54
# [Azure ML CLI](#tab/cli)
55
55
56
56
```azurecli
57
-
az ml workspace show --query storage_account
57
+
DATASTORE_ID=$(az ml datastore show -n workspaceblobstore | jq -r '.id')
58
58
```
59
+
60
+
> [!NOTE]
61
+
> Data stores ID would look like `azureml:/subscriptions/<subscription>/resourceGroups/<resource-group>/providers/Microsoft.MachineLearningServices/workspaces/<workspace>/datastores/<data-store>`.
59
62
60
63
# [Azure ML SDK for Python](#tab/sdk)
61
64
@@ -66,21 +69,25 @@ Data from Azure Machine Learning registered data stores can be directly referenc
66
69
# [REST](#tab/rest)
67
70
68
71
Use the Azure ML CLI, Azure ML SDK for Python, or Studio to get the data store information.
72
+
73
+
---
74
+
75
+
> [!TIP]
76
+
> The default blob data store in a workspace is called __workspaceblobstore__. You can skip this step if you already know the resource ID of the default data store in your workspace.
69
77
70
-
1. We'll need to upload some sample data to it. This example assumes you've uploaded the sample data included in the repo in the folder `sdk/python/endpoints/batch/heart-classifier/data` in the folder `heart-classifier/data` in the blob storage account.
78
+
1. We'll need to upload some sample data to it. This example assumes you've uploaded the sample data included in the repo in the folder `sdk/python/endpoints/batch/heart-classifier/data` in the folder `heart-classifier/data` in the blob storage account. Ensure you have done that before moving forward.
71
79
72
80
1. Create a data input:
73
81
74
82
# [Azure ML CLI](#tab/cli)
83
+
84
+
Let's place the file path in the following variable:
75
85
76
86
```azurecli
77
87
DATA_PATH="heart-disease-uci-unlabeled"
78
-
DATASTORE_ID=$(az ml workspace show | jq -r '.storage_account')
88
+
INPUT_PATH="$DATASTORE_ID/paths/$DATA_PATH"
79
89
```
80
90
81
-
> [!TIP]
82
-
> You can skip this step if you already know the name of the data store you want to use. Here it is used only to know the name of the default data store of the workspace.
83
-
84
91
# [Azure ML SDK for Python](#tab/sdk)
85
92
86
93
```python
@@ -90,23 +97,23 @@ Data from Azure Machine Learning registered data stores can be directly referenc
90
97
91
98
# [REST](#tab/rest)
92
99
93
-
Use the Azure ML CLI, Azure ML SDK for Python, or Studio to get the data store information.
100
+
Use the Azure ML CLI, Azure ML SDK for Python, or Studio to get the subscription ID, resource group, workspace, and name of the data store. You will need them later.
101
+
94
102
---
95
-
103
+
96
104
> [!NOTE]
97
-
> Data stores ID would look like `/subscriptions/<subscription>/resourcegroups/<resource-group>/providers/microsoft.storage/storageaccounts/<storage-account-name>`.
105
+
> See how the path `paths` is appended to the resource id of the data store to indicate that what follows is a path inside of it.
98
106
107
+
> [!TIP]
108
+
> You can also use `azureml:/datastores/<data-store>/paths/<data-path>` as a way to indicate the input.
99
109
100
110
1. Run the deployment:
101
111
102
112
# [Azure ML CLI](#tab/cli)
103
113
104
114
```bash
105
-
INVOKE_RESPONSE = $(az ml batch-endpoint invoke --name $ENDPOINT_NAME --input $DATASTORE_ID/paths/$DATA_PATH)
115
+
INVOKE_RESPONSE = $(az ml batch-endpoint invoke --name $ENDPOINT_NAME --input $INPUT_PATH)
106
116
```
107
-
108
-
> [!TIP]
109
-
> You can also use `--input azureml:/datastores/<data_store_name>/paths/<data_path>` as a way to indicate the input.
110
117
111
118
# [Azure ML SDK for Python](#tab/sdk)
112
119
@@ -127,7 +134,7 @@ Data from Azure Machine Learning registered data stores can be directly referenc
@@ -136,7 +143,7 @@ Data from Azure Machine Learning registered data stores can be directly referenc
136
143
137
144
## Reading data from a data asset
138
145
139
-
Azure Machine Learning data assets (formaly known as datasets) are supported as inputs for jobs. Follow these steps to run a batch endpoint job using data stored in a registered data asset in Azure Machine Learning:
146
+
Azure Machine Learning data assets (formerly known as datasets) are supported as inputs for jobs. Follow these steps to run a batch endpoint job using data stored in a registered data asset in Azure Machine Learning:
140
147
141
148
> [!WARNING]
142
149
> Data assets of type Table (`MLTable`) aren't currently supported.
@@ -185,8 +192,10 @@ Azure Machine Learning data assets (formaly known as datasets) are supported as
185
192
1. Create a data input:
186
193
187
194
# [Azure ML CLI](#tab/cli)
188
-
195
+
196
+
```azurecli
189
197
DATASET_ID=$(az ml data show -n heart-dataset-unlabeled --label latest --query id)
198
+
```
190
199
191
200
# [Azure ML SDK for Python](#tab/sdk)
192
201
@@ -201,7 +210,7 @@ Azure Machine Learning data assets (formaly known as datasets) are supported as
201
210
---
202
211
203
212
> [!NOTE]
204
-
> Data stores ID would look like `/subscriptions/<subscription>/resourcegroups/<resource-group>/providers/microsoft.storage/storageaccounts/<storage-account-name>`.
213
+
> Data assets ID would look like `/subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.MachineLearningServices/workspaces/<workspace>/data/<data-asset>/versions/<version>`.
0 commit comments