You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this article, you learn how to import your own data in the designer to create custom solutions. There are two ways you can import data into the designer:
20
+
In this article, you learn how to import your own data into the Azure Machine Learning designer to create custom solutions. There are two ways you can import data into the designer:
21
21
22
-
***Azure Machine Learning datasets** - Register [datasets](concept-data.md) in Azure Machine Learning to enable advanced features that help you manage your data.
23
-
***Import Data component** - Use the [Import Data](../algorithm-module-reference/import-data.md) component to directly access data from online data sources.
22
+
***Azure Machine Learning datasets**: Register [datasets](concept-data.md#reference-data-in-storage-with-datasets) in Azure Machine Learning to enable advanced features that help you manage your data.
23
+
***Import Data component**: Use the [Import Data](../algorithm-module-reference/import-data.md) component to directly access data from online data sources.
24
24
25
25
> [!IMPORTANT]
26
-
> If you do not see graphical elements mentioned in this document, such as buttons in studio or designer, you may not have the right level of permissions to the workspace. Please contact your Azure subscription administrator to verify that you have been granted the correct level of access. For more information, see [Manage users and roles](../how-to-assign-roles.md).
26
+
> If you don't see graphical elements mentioned in this document, such as buttons in studio or designer, you might not have the right level of permissions to the workspace. Contact your Azure subscription administrator to verify that you have been granted the correct level of access. For more information, see [Manage users and roles](../how-to-assign-roles.md).
27
27
28
28
## Use Azure Machine Learning datasets
29
29
30
-
We recommend that you use [datasets](concept-data.md) to import data into the designer. When you register a dataset, you can take full advantage of advanced data features like [versioning and tracking](how-to-version-track-datasets.md) and [data monitoring](how-to-monitor-datasets.md).
30
+
We recommend that you use [datasets](concept-data.md#reference-data-in-storage-with-datasets) to import data into the designer. When you register a dataset, you can take full advantage of advanced data features like [versioning and tracking](how-to-version-track-datasets.md) and [data monitoring](how-to-monitor-datasets.md).
31
31
32
32
### Register a dataset
33
33
34
-
You can register existing datasets [programmatically with the SDK](how-to-create-register-datasets.md#create-datasets-from-datastores) or [visually in Azure Machine Learning studio](how-to-connect-data-ui.md#create-data-assets).
34
+
You can register existing datasets [programmatically with the SDK](how-to-create-register-datasets.md#create-datasets-from-datastores) or [visually in the Azure Machine Learning studio](how-to-connect-data-ui.md#create-data-assets).
35
35
36
36
You can also register the output for any designer component as a dataset.
37
37
38
38
1. Select the component that outputs the data you want to register.
39
39
40
-
1. In the properties pane, select **Outputs + logs** > **Register dataset**.
40
+
1. In the properties pane, select **Outputs + logs** > **Data outputs** > **Register data**.
41
41
42
-
42
+
:::image type="content" source="media/how-to-designer-import-data/register-dataset-designer.png" alt-text="Screenshot showing how to navigate to the Register Dataset option.":::
43
43
44
44
If the component output data is in a tabular format, you must choose to register the output as a **file dataset** or **tabular dataset**.
45
45
46
-
-**File dataset** registers the component's output folder as a file dataset. The output folder contains a data file and meta files that the designer uses internally. Select this option if you want to continue to use the registered dataset in the designer.
46
+
-**File dataset** registers the component's output folder as a file dataset. The output folder contains a data file and meta files that the designer uses internally. Select this option if you want to continue to use the registered dataset in the designer.
47
47
48
-
-**Tabular dataset** registers only the component's the output data file as a tabular dataset. This format is easily consumed by other tools, for example in Automated Machine Learning or the Python SDK. Select this option if you plan to use the registered dataset outside of the designer.
48
+
-**Tabular dataset** registers only the component's the output data file as a tabular dataset. This format is easily consumed by other tools, for example in Automated Machine Learning or the Python SDK. Select this option if you plan to use the registered dataset outside of the designer.
49
49
50
50
51
51
### Use a dataset
52
52
53
-
Your registered datasets can be found in the component palette, under **Datasets**. To use a dataset, drag and drop it onto the pipeline canvas. Then, connect the output port of the dataset to other components in the canvas.
53
+
Your registered datasets can be found in the component palette, under **Data**. To use a dataset, drag and drop it onto the pipeline canvas. Then, connect the output port of the dataset to other components in the canvas.
54
54
55
55
If you register a file dataset, the output port type of the dataset is **AnyDirectory**. If you register a Tabular dataset, the output port type of the dataset if **DataFrameDirectory**. Note that if you connect the output port of the dataset to other components in the designer, the port type of datasets and components need to be aligned.
56
56
@@ -62,37 +62,38 @@ If you register a file dataset, the output port type of the dataset is **AnyDire
62
62
63
63
### Limitations
64
64
65
-
- Currently you can only visualize tabular dataset in the designer. If you register a file dataset outside designer, you cannot visualize it in the designer canvas.
66
-
- Currently the designer only supports preview outputs which are stored in **Azure blob storage**. You can check and change your output datastore in the **Output settings** under **Parameters** tab in the right panel of the component.
67
-
- If your data is stored in virtual network (VNet) and you want to preview, you need to enable workspace managed identity of the datastore.
68
-
1. Go the related datastore and click**Update authentication**
- Currently you can only visualize a tabular dataset in the designer. If you register a file dataset outside designer, you can't visualize it in the designer canvas.
66
+
- Currently the designer only supports preview outputs that are stored in **Azure blob storage**. You can check and change your output datastore in the **Output settings** under **Parameters** tab in the right panel of the component.
67
+
- If your data is stored in a virtual network and you want to preview, you need to enable workspace managed identity of the datastore.
68
+
1. Go the related datastore and select**Update authentication**.
69
+
:::image type="content" source="../media/resource-known-issues/datastore-update-credential.png" alt-text="Screenshot that shows how to update credentials.":::
70
+
1. Select the toggle switch to use workspace managed identity.
71
+
:::image type="content" source="../media/resource-known-issues/enable-workspace-managed-identity.png" alt-text="Screenshot that shows how to Enable Workspace Managed Identity.":::
72
72
73
73
## Import data using the Import Data component
74
74
75
-
While we recommend that you use datasets to import data, you can also use the [Import Data](../algorithm-module-reference/import-data.md) component. The Import Data component skips registering your dataset in Azure Machine Learning and imports data directly from a [datastore](concept-data.md) or HTTP URL.
75
+
While we recommend that you use datasets to import data, you can also use the [Import Data](../algorithm-module-reference/import-data.md) component. The **Import Data** component skips registering your dataset in Azure Machine Learning and imports data directly from a [datastore](concept-data.md) or HTTP URL.
76
76
77
77
For detailed information on how to use the Import Data component, see the [Import Data reference page](../algorithm-module-reference/import-data.md).
78
78
79
79
> [!NOTE]
80
-
> If your dataset has too many columns, you may encounter the following error: "Validation failed due to size limitation". To avoid this, [register the dataset in the Datasets interface](how-to-connect-data-ui.md#create-data-assets).
80
+
> If your dataset has too many columns, you might encounter the following error: "Validation failed due to size limitation". To avoid this, [register the dataset in the Datasets interface](how-to-connect-data-ui.md#create-data-assets).
81
81
82
82
## Supported sources
83
83
84
-
This section lists the data sources supported by the designer. Data comes into the designer from either a datastore or from [tabular dataset](how-to-create-register-datasets.md#dataset-types).
84
+
This section lists the data sources supported by the designer. Data comes into the designer from either a datastore or from a [tabular dataset](how-to-create-register-datasets.md#dataset-types).
85
85
86
86
### Datastore sources
87
+
87
88
For a list of supported datastore sources, see [Access data in Azure storage services](how-to-access-data.md#supported-data-storage-service-types).
88
89
89
90
### Tabular dataset sources
90
91
91
92
The designer supports tabular datasets created from the following sources:
92
-
* Delimited files
93
-
* JSON files
94
-
* Parquet files
95
-
* SQL queries
93
+
* Delimited files
94
+
* JSON files
95
+
* Parquet files
96
+
* SQL queries
96
97
97
98
## Data types
98
99
@@ -112,8 +113,9 @@ Modules in the designer are limited by the size of the compute target. For large
112
113
113
114
## Access data in a virtual network
114
115
115
-
If your workspace is in a virtual network, you must perform additional configuration steps to visualize data in the designer. For more information on how to use datastores and datasets in a virtual network, see [Use Azure Machine Learning studio in an Azure virtual network](../how-to-enable-studio-virtual-network.md).
116
+
If your workspace is in a virtual network, you must perform additional configuration steps to visualize data in the designer. For more information on how to use datastores and datasets in a virtual network, see [Use Azure Machine Learning studio in an Azure virtual network](../how-to-enable-studio-virtual-network.md).
116
117
117
-
## Next steps
118
+
## Next step
118
119
119
-
Learn the designer fundamentals with this [Tutorial: Predict automobile price with the designer](tutorial-designer-automobile-price-train-score.md).
120
+
> [!div class="nextstepaction"]
121
+
> [Tutorial: Train a no-code regression model using designer](tutorial-designer-automobile-price-train-score.md)
description: Monitor your Azure Machine Learning designer experiments. Enable logging using the Execute Python Script component and view the logged results in the studio.
In this article, you learn how to add logging code to designer pipelines. You also learn how to view those logs using the Azure Machine Learning studio web portal.
20
+
In this article, you learn how to add code to designer pipelines to enable log metrics. You also learn how to view those logs using the Azure Machine Learning studio web portal.
21
21
22
-
For more information on logging metrics using the SDK authoring experience, see [Monitor Azure Machine Learning experiment runs and metrics](../how-to-log-view-metrics.md).
22
+
For more information on logging metrics using the SDK authoring experience, see [Log & view metrics and log files](../how-to-log-view-metrics.md).
23
23
24
24
## Enable logging with Execute Python Script
25
25
26
-
Use the [Execute Python Script](../algorithm-module-reference/execute-python-script.md) component to enable logging in designer pipelines. Although you can log any value with this workflow, it's especially useful to log metrics from the __Evaluate Model__ component to track model performance across runs.
26
+
Use the [Execute Python Script](../algorithm-module-reference/execute-python-script.md) component to enable logging in designer pipelines. Although you can log any value with this workflow, it's especially useful to log metrics from the **Evaluate Model** component to track model performance across runs.
27
27
28
-
The following example shows you how to log the mean squared error of two trained models using the Evaluate Model and Execute Python Script components.
28
+
The following example shows how to log the mean squared error of two trained models using the **Evaluate Model** and **Execute Python Script** components.
29
29
30
-
1. Connect an __Execute Python Script__ component to the output of the __Evaluate Model__ component.
30
+
1. Connect an **Execute Python Script** component to the output of the **Evaluate Model** component.
31
31
32
-
32
+
:::image type="content" source="../media/how-to-log-view-metrics/designer-logging-pipeline.png" alt-text="Screenshot that shows how to connect the Execute Python Script component to the Evaluate Model component." lightbox="../media/how-to-log-view-metrics/designer-logging-pipeline.png":::
33
33
34
-
1. Paste the following code into the __Execute Python Script__ code editor to log the mean absolute error for your trained model. You can use a similar pattern to log any other value in the designer:
34
+
1. Paste the following code into the **Execute Python Script** code editor to log the mean absolute error for your trained model. You can use a similar pattern to log any other value in the designer:
@@ -56,27 +56,27 @@ The following example shows you how to log the mean squared error of two trained
56
56
57
57
return dataframe1,
58
58
```
59
-
60
-
This code uses the Azure Machine Learning Python SDK to log values. It uses Run.get_context() to get the context of the current run. It then logs values to that context with the run.parent.log() method. It uses `parent` to log values to the parent pipeline run rather than the component run.
61
59
62
-
For more information on how to use the Python SDK to log values, see [Enable logging in Azure Machine Learning training runs](../how-to-log-view-metrics.md).
60
+
This code uses the Azure Machine Learning Python SDK to log values. It uses `Run.get_context()` to get the context of the current run. It then logs values to that context with the `run.parent.log()` method. It uses `parent` to log values to the parent pipeline run rather than the component run.
61
+
62
+
For more information on how to use the Python SDK to log values, see [Log & view metrics and log files](../how-to-log-view-metrics.md).
63
63
64
64
## View logs
65
65
66
66
After the pipeline run completes, you can see the *Mean_Absolute_Error*in the Experiments page.
67
67
68
68
1. Navigate to the **Jobs** section.
69
+
69
70
1. Select your experiment.
70
-
1. Select the job in your experiment you want to view.
71
-
1. Select **Metrics**.
72
71
73
-
72
+
1. Select the job in your experiment that you want to view.
74
73
75
-
## Next steps
74
+
1. Select **Metrics**.
76
75
77
-
In this article, you learned how to use logs in the designer. For next steps, see these related articles:
76
+
:::image type="content" source="../media/how-to-log-view-metrics/experiment-page-metrics-across-runs.png" alt-text="Screenshot that shows job metrics in the studio." lightbox="../media/how-to-log-view-metrics/experiment-page-metrics-across-runs.png":::
78
77
78
+
## Related content
79
79
80
-
*Learn how to troubleshoot designer pipelines, see [Debug & troubleshoot MLpipelines](how-to-debug-pipelines.md#azure-machine-learning-designer).
81
-
*Learn how to use the Python SDK to log metrics in the SDK authoring experience, see [Enable logging in Azure Machine Learning training runs](how-to-log-view-metrics.md).
82
-
*Learn how to use [Execute Python Script](../algorithm-module-reference/execute-python-script.md)in the designer.
0 commit comments