Skip to content

Commit 0534d04

Browse files
Merge pull request #220328 from SturgeonMi/patch-18
Update URI support in Data
2 parents e4d9846 + fcbc878 commit 0534d04

File tree

2 files changed

+27
-24
lines changed

2 files changed

+27
-24
lines changed

articles/machine-learning/how-to-create-data-assets.md

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -49,12 +49,12 @@ To create and work with data assets, you need:
4949
When you create a data asset in Azure Machine Learning, you'll need to specify a `path` parameter that points to its location. Below is a table that shows the different data locations supported in Azure Machine Learning and examples for the `path` parameter:
5050

5151

52-
|Location | Examples |
53-
|---------|---------|
54-
|A path on your local computer | `./home/username/data/my_data` |
55-
|A path on a datastore | `azureml://datastores/<data_store_name>/paths/<path>` |
56-
|A path on a public http(s) server | `https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv` |
57-
|A path on Azure Storage |`wasbs://<containername>@<accountname>.blob.core.windows.net/<path_to_data>/` <br> `abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>` <br> `adl://<accountname>.azuredatalakestore.net/<path_to_data>/`<br> `https://<account_name>.blob.core.windows.net/<container_name>/path` |
52+
|Location | Examples | Notes
53+
|---------|---------|---------|
54+
|A path on your local computer | `./home/username/data/my_data` ||
55+
|A path on a datastore | `azureml://datastores/<data_store_name>/paths/<path>` | |
56+
|A path on a public http(s) server | `https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv` | https path pointing to a folder is not supported since https is not a filesystem. Please use other formats(wasbs/abfss/adl) instead for folder type of data.|
57+
|A path on Azure Storage |`wasbs://<containername>@<accountname>.blob.core.windows.net/<path_to_data>/` <br> `abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>` <br> `adl://<accountname>.azuredatalakestore.net/<path_to_data>/` ||
5858

5959

6060
> [!NOTE]
@@ -63,6 +63,9 @@ When you create a data asset in Azure Machine Learning, you'll need to specify a
6363
> [!IMPORTANT]
6464
> The studio only supports browsing of credential-less ADLS Gen 2 datastores.
6565
66+
> [!IMPORTANT]
67+
> Authentication to access data will use user's identity or compute MSI by default.
68+
6669

6770
## Data asset types
6871
- [**URIs**](#Create a `uri_folder` data asset) - A **U**niform **R**esource **I**dentifier that is a reference to a storage location on your local computer or in the cloud that makes it easy to access data in your jobs. Azure Machine Learning distinguishes two types of URIs:`uri_file` and `uri_folder`.
@@ -89,7 +92,7 @@ $schema: https://azuremlschemas.azureedge.net/latest/data.schema.json
8992

9093
# Supported paths include:
9194
# local: ./<path>
92-
# blob: https://<account_name>.blob.core.windows.net/<container_name>/<path>
95+
# blob: wasbs://<containername>@<accountname>.blob.core.windows.net/<path>/
9396
# ADLS gen2: abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>/
9497
# Datastore: azureml://datastores/<data_store_name>/paths/<path>
9598
type: uri_folder
@@ -114,7 +117,7 @@ from azure.ai.ml.constants import AssetTypes
114117

115118
# Supported paths include:
116119
# local: './<path>'
117-
# blob: 'https://<account_name>.blob.core.windows.net/<container_name>/<path>'
120+
# blob: 'wasbs://<containername>@<accountname>.blob.core.windows.net/<path>/'
118121
# ADLS gen2: 'abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>/'
119122
# Datastore: 'azureml://datastores/<data_store_name>/paths/<path>'
120123

@@ -162,7 +165,7 @@ $schema: https://azuremlschemas.azureedge.net/latest/data.schema.json
162165

163166
# Supported paths include:
164167
# local: ./<path>/<file>
165-
# blob: https://<account_name>.blob.core.windows.net/<container_name>/<path>/<file>
168+
# blob: wasbs://<containername>@<accountname>.blob.core.windows.net/<path>/<file>
166169
# ADLS gen2: abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>/<file>
167170
# Datastore: azureml://datastores/<data_store_name>/paths/<path>/<file>
168171

@@ -183,7 +186,7 @@ from azure.ai.ml.constants import AssetTypes
183186

184187
# Supported paths include:
185188
# local: './<path>/<file>'
186-
# blob: 'https://<account_name>.blob.core.windows.net/<container_name>/<path>/<file>'
189+
# blob: 'wasbs://<containername>@<accountname>.blob.core.windows.net/<path>/<file>'
187190
# ADLS gen2: 'abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>/<file>'
188191
# Datastore: 'azureml://datastores/<data_store_name>/paths/<path>/<file>'
189192
my_path = '<path>'
@@ -296,7 +299,7 @@ $schema: https://azuremlschemas.azureedge.net/latest/data.schema.json
296299
# path must point to **folder** containing MLTable artifact (MLTable file + data
297300
# Supported paths include:
298301
# local: ./<path>
299-
# blob: https://<account_name>.blob.core.windows.net/<container_name>/<path>
302+
# blob: wasbs://<containername>@<accountname>.blob.core.windows.net/<path>/
300303
# ADLS gen2: abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>/
301304
# Datastore: azureml://datastores/<data_store_name>/paths/<path>
302305
@@ -326,7 +329,7 @@ from azure.ai.ml.constants import AssetTypes
326329
# my_path must point to folder containing MLTable artifact (MLTable file + data
327330
# Supported paths include:
328331
# local: './<path>'
329-
# blob: 'https://<account_name>.blob.core.windows.net/<container_name>/<path>'
332+
# blob: 'wasbs://<containername>@<accountname>.blob.core.windows.net/<path>/'
330333
# ADLS gen2: 'abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>/'
331334
# Datastore: 'azureml://datastores/<data_store_name>/paths/<path>'
332335

articles/machine-learning/how-to-read-write-data-v2.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -37,13 +37,13 @@ Learn how to read and write data for your jobs with the Azure Machine Learning P
3737
When you provide a data input/output to a Job, you'll need to specify a `path` parameter that points to the data location. Below is a table that shows the different data locations supported in Azure Machine Learning and examples for the `path` parameter:
3838

3939

40-
|Location | Examples |
41-
|---------|---------|
42-
|A path on your local computer | `./home/username/data/my_data` |
43-
|A path on a public http(s) server | `https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv` |
44-
|A path on Azure Storage | `https://<account_name>.blob.core.windows.net/<container_name>/<path>` <br> `abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>` |
45-
|A path on a Datastore | `azureml://datastores/<data_store_name>/paths/<path>` |
46-
|A path to a Data Asset | `azureml:<my_data>:<version>` |
40+
|Location | Examples | Notes|
41+
|---------|---------|---------|
42+
|A path on your local computer | `./home/username/data/my_data` ||
43+
|A path on a public http(s) server | `https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv` | https path pointing to a folder is not supported since https is not a filesystem. Please use other formats(wasbs/abfss/adl) instead for folder type of data.|
44+
|A path on Azure Storage | `wasbs://<containername>@<accountname>.blob.core.windows.net/<path_to_data>/` <br> `abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>` <br> `adl://<accountname>.azuredatalakestore.net/<path_to_data>/` ||
45+
|A path on a Datastore | `azureml://datastores/<data_store_name>/paths/<path>` ||
46+
|A path to a Data Asset | `azureml:<my_data>:<version>` ||
4747

4848
## Supported modes
4949

@@ -75,7 +75,7 @@ Create a job specification YAML file (`<file-name>.yml`). Specify in the `inputs
7575
$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json
7676

7777
# Possible Paths for Data:
78-
# Blob: https://<account_name>.blob.core.windows.net/<container_name>/<folder>/<file>
78+
# Blob: wasbs://<containername>@<accountname>.blob.core.windows.net/<folder>/<file>
7979
# Datastore: azureml://datastores/paths/<folder>/<file>
8080
# Data Asset: azureml:<my_data>:<version>
8181

@@ -118,7 +118,7 @@ ml_client = MLClient.from_config()
118118
# AssetTypes.MLTABLE
119119

120120
# Possible Paths for Data:
121-
# Blob: https://<account_name>.blob.core.windows.net/<container_name>/<folder>/<file>
121+
# Blob: wasbs://<containername>@<accountname>.blob.core.windows.net/<folder>/<file>
122122
# Datastore: azureml://datastores/paths/<folder>/<file>
123123
# Data Asset: azureml:<my_data>:<version>
124124

@@ -290,7 +290,7 @@ Create a job specification YAML file (`<file-name>.yml`), with the `outputs` sec
290290
$schema: https://azuremlschemas.azureedge.net/latest/CommandJob.schema.json
291291

292292
# Possible Paths for Data:
293-
# Blob: https://<account_name>.blob.core.windows.net/<container_name>/<folder>/<file>
293+
# Blob: wasbs://<containername>@<accountname>.blob.core.windows.net/<folder>/<file>
294294
# Datastore: azureml://datastores/paths/<folder>/<file>
295295
# Data Asset: azureml:<my_data>:<version>
296296

@@ -331,7 +331,7 @@ from azure.ai.ml.constants import AssetTypes
331331
# AssetTypes.MLTABLE
332332

333333
# Possible Paths for Data:
334-
# Blob: https://<account_name>.blob.core.windows.net/<container_name>/<folder>/<file>
334+
# Blob: wasbs://<containername>@<accountname>.blob.core.windows.net/<folder>/<file>
335335
# Datastore: azureml://datastores/paths/<folder>/<file>
336336
# Data Asset: azureml:<my_data>:<version>
337337

@@ -386,4 +386,4 @@ The following example defines a pipeline containing three nodes and moves data b
386386

387387
* [Train models](how-to-train-model.md)
388388
* [Tutorial: Create production ML pipelines with Python SDK v2](tutorial-pipeline-python-sdk.md)
389-
* Learn more about [Data in Azure Machine Learning](concept-data.md)
389+
* Learn more about [Data in Azure Machine Learning](concept-data.md)

0 commit comments

Comments
 (0)