You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-workspace.md
+70-82Lines changed: 70 additions & 82 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,57 +24,104 @@ Ready to get started? [Create a workspace](#create-a-workspace).
24
24
25
25
:::image type="content" source="./media/concept-workspace/workspace.png" alt-text="Screenshot of the Azure Machine Learning workspace.":::
26
26
27
-
## Working with workspaces
27
+
## Tasks performed within a workspace
28
+
29
+
For machine learning teams, the workspace is a place to organize their work. Below are some of the tasks you can start from a workspace:
28
30
29
-
For machine learning teams, the workspace is a place to organize their work. To administrators, workspaces serve as containers for access management, cost management and data isolation. Below are some tips for working with workspaces:
31
+
+[Create jobs](how-to-train-model.md) - Jobs are training runs you use to build your models. You can group jobs into [experiments](how-to-log-view-metrics.md) to compare metrics.
32
+
+[Author pipelines](concept-ml-pipelines.md) - Pipelines are reusable workflows for training and retraining your model.
33
+
+[Register data assets](concept-data.md) - Data assets aid in management of the data you use for model training and pipeline creation.
34
+
+[Register models](how-to-log-mlflow-models.md) - Once you have a model you want to deploy, you create a registered model.
35
+
:::moniker range="azureml-api-2"
36
+
+[Create online endpoints](concept-endpoints.md) - Use a registered model and a scoring script to create an online endpoint.
37
+
:::moniker-end
38
+
:::moniker range="azureml-api-1"
39
+
+[Deploy a model](./v1/how-to-deploy-and-where.md) - Use the registered model and a scoring script to deploy a model.
40
+
:::moniker-end
41
+
42
+
Besides grouping your machine learning results, workspaces also host resource configurations:
43
+
44
+
+[Compute targets](concept-compute-target.md) are used to run your experiments.
45
+
+[Datastores](how-to-datastore.md) define how you and others can connect to data sources when using data assets.
46
+
+[Security settings](tutorial-create-secure-workspace.md) - Networking, identity and access control, and encryption settings.
47
+
48
+
## Organizing workspaces
49
+
50
+
For machine learning team leads and administrators, workspaces serve as containers for access management, cost management and data isolation. Below are some tips for organizing workspaces:
30
51
31
52
+**Use [user roles](how-to-assign-roles.md)** for permission management in the workspace between users. For example a data scientist, a machine learning engineer or an admin.
32
-
+**Assign access to user groups**: By using Azure Active Directory user groups, you don't have to add individual users to each workspace and other resources the same group of users requires access to.
53
+
+**Assign access to user groups**: By using Azure Active Directory user groups, you don't have to add individual users to each workspace, and to other resources the same group of users requires access to.
33
54
+**Create a workspace per project**: While a workspace can be used for multiple projects, limiting it to one project per workspace allows for cost reporting accrued to a project level. It also allows you to manage configurations like datastores in the scope of each project.
34
55
+**Share Azure resources**: Workspaces require you to create several [associated resources](#associated-resources). Share these resources between workspaces to save repetitive setup steps.
35
56
+**Enable self-serve**: Pre-create and secure [associated resources](#associated-resources) as an IT admin, and use [user roles](how-to-assign-roles.md) to let data scientists create workspaces on their own.
36
57
+**Share assets**: You can share assets between workspaces using [Azure Machine Learning registries (preview)](how-to-share-models-pipelines-across-workspaces-with-registries.md).
37
58
38
-
## What content is stored in a workspace?
59
+
## How is my content stored in a workspace?
39
60
40
-
Your workspace keeps a history of all training runs, with logs, metrics, output, lineage metadata, and a snapshot of your scripts. As you perform tasks in Azure Machine Learning, artifacts are generated. Their metadata and data are stored in the workspace and on its [associated resources](#associated-resources).
61
+
Your workspace keeps a history of all training runs, with logs, metrics, output, lineage metadata, and a snapshot of your scripts. As you perform tasks in Azure Machine Learning, artifacts are generated. Their metadata and data are stored in the workspace and on its associated resources.
41
62
42
-
## Tasks performed within a workspace
63
+
## Associated resources
43
64
44
-
The following constructs you can find and manage within the workspace boundary.
65
+
When you create a new workspace, you're required to bring other Azure resources to store your data. If not provided by you, these resources will automatically be created by Azure Machine Learning.
45
66
46
-
+[Compute targets](concept-compute-target.md) are used to run your experiments.
47
-
+ Jobs are training runs you use to build your models. You can organize your jobs into Experiments.
48
-
+[Pipelines](concept-ml-pipelines.md) are reusable workflows for training and retraining your model.
49
-
+[Data assets](concept-data.md) aid in management of the data you use for model training and pipeline creation.
50
-
+ Once you have a model you want to deploy, you create a registered model.
67
+
+[Azure Storage account](https://azure.microsoft.com/services/storage/). Stores machine learning artifacts such as job logs. By default, this storage account is used when you upload data to the workspace. Jupyter notebooks that are used with your Azure Machine Learning compute instances are stored here as well.
68
+
69
+
> [!IMPORTANT]
70
+
> To use an existing Azure Storage account, it can't be of type BlobStorage, a premium account (Premium_LRS and Premium_GRS) and cannot have a hierarchical namespace (used with Azure Data Lake Storage Gen2). You can use premium storage or hierarchical namespace as additional storage by [creating a datastore](how-to-datastore.md).
71
+
> Do not enable hierarchical namespace on the storage account after upgrading to general-purpose v2.
72
+
> If you bring an existing general-purpose v1 storage account, you may [upgrade this to general-purpose v2](../storage/common/storage-account-upgrade.md) after the workspace has been created.
73
+
74
+
+[Azure Container Registry](https://azure.microsoft.com/services/container-registry/). Stores created docker containers, when you build custom environments via Azure Machine Learning. Scenarios that trigger creation of custom environments include AutoML when deploying models and data profiling.
75
+
76
+
> [!NOTE]
77
+
> Workspaces can be created without Azure Container Registry as a dependency if you do not have a need to build custom docker containers. To read container images, Azure Machine Learning also works with external container registries. Azure Container Registry is automatically provisioned when you build custom docker images. Use Azure RBAC to prevent customer docker containers from being built.
78
+
79
+
> [!NOTE]
80
+
> If your subscription setting requires adding tags to resources under it, Azure Container Registry (ACR) created by Azure Machine Learning will fail, since we cannot set tags to ACR.
81
+
82
+
+[Azure Application Insights](https://azure.microsoft.com/services/application-insights/). Helps you monitor and collect diagnostic information from your inference endpoints.
83
+
:::moniker range="azureml-api-2"
84
+
For more information, see [Monitor online endpoints](how-to-monitor-online-endpoints.md).
85
+
:::moniker-end
86
+
87
+
+[Azure Key Vault](https://azure.microsoft.com/services/key-vault/). Stores secrets that are used by compute targets and other sensitive information that's needed by the workspace.
88
+
89
+
## Create a workspace
90
+
91
+
There are multiple ways to create a workspace. To get started use one of the following options:
92
+
93
+
* The [Azure Machine Learning studio](quickstart-create-resources.md) lets you quickly create a workspace with default settings.
94
+
* Use [Azure portal](how-to-manage-workspace.md?tabs=azure-portal#create-a-workspace) for a point-and-click interface with more security options.
95
+
* Use the [VS Code extension](how-to-manage-resources-vscode.md#create-a-workspace) if you work in Visual Studio Code.
96
+
97
+
To automate workspace creation using your preferred security settings:
98
+
*[Azure Resource Manager / Bicep templates](how-to-create-workspace-template.md) provide a declarative syntax to deploy Azure resources. An alternative option is to use [Terraform](how-to-manage-workspace-terraform.md). Also see [How to create a secure workspace by using a template](tutorial-create-secure-workspace-template.md).
51
99
:::moniker range="azureml-api-2"
52
-
+ Use the registered model and a scoring script to create an [online endpoint](concept-endpoints.md).
100
+
* Use the [Azure Machine Learning CLI](how-to-configure-cli.md) or [Azure Machine Learning SDK for Python](how-to-manage-workspace.md?tabs=python#create-a-workspace) for prototyping and as part of your [MLOps workflows](concept-model-management-and-deployment.md).
53
101
:::moniker-end
54
102
:::moniker range="azureml-api-1"
55
-
+ Use the registered model and a scoring script to [deploy the model](./v1/how-to-deploy-and-where.md)
103
+
* Use the [Azure Machine Learning CLI](./v1/reference-azure-machine-learning-cli.md) or [Azure Machine Learning SDK for Python](how-to-manage-workspace.md?tabs=python#create-a-workspace) for prototyping and as part of your [MLOps workflows](concept-model-management-and-deployment.md).
56
104
:::moniker-end
105
+
* Use [REST APIs](how-to-manage-rest.md) directly in scripting environment, for platform integration or in MLOps workfows.
57
106
58
-
## Tools for workspace interaction
107
+
## Tools for workspace interaction and management
59
108
60
-
You can interact with your workspace in the following ways:
109
+
Once your workspace is set up, you can interact with it in the following ways:
61
110
62
111
+ On the web:
63
112
+[Azure Machine Learning studio ](https://ml.azure.com)
@@ -86,74 +133,15 @@ You can also perform the following workspace management tasks:
86
133
> [!WARNING]
87
134
> Moving your Azure Machine Learning workspace to a different subscription, or moving the owning subscription to a new tenant, is not supported. Doing so may cause errors.
88
135
89
-
## Create a workspace
90
-
91
-
There are multiple ways to create a workspace:
92
-
93
-
* Use [Azure Machine Learning studio](quickstart-create-resources.md) to quickly create a workspace with default settings.
94
-
* Use the [Azure portal](how-to-manage-workspace.md?tabs=azure-portal#create-a-workspace) for a point-and-click interface with more options.
95
-
* Use the [Azure Machine Learning SDK for Python](how-to-manage-workspace.md?tabs=python#create-a-workspace) to create a workspace on the fly from Python scripts or Jupyter notebooks.
96
-
:::moniker range="azureml-api-2"
97
-
* Use an [Azure Resource Manager template](how-to-create-workspace-template.md) or the [Azure Machine Learning CLI](how-to-configure-cli.md) when you need to automate or customize the creation with corporate security standards.
98
-
:::moniker-end
99
-
:::moniker range="azureml-api-1"
100
-
* Use an [Azure Resource Manager template](how-to-create-workspace-template.md) or the [Azure Machine Learning CLI](./v1/reference-azure-machine-learning-cli.md) when you need to automate or customize the creation with corporate security standards.
101
-
:::moniker-end
102
-
* If you work in Visual Studio Code, use the [VS Code extension](how-to-manage-resources-vscode.md#create-a-workspace).
103
-
104
-
> [!NOTE]
105
-
> The workspace name is case-insensitive.
106
-
107
136
## Sub resources
108
137
109
-
These sub resources are the main resources that are made in the Azure Machine Learning workspace.
138
+
When you create compute clusters and compute instances in Azure Machine Learning, sub resources are created.
110
139
111
-
* VMs: provide computing power for your Azure Machine Learning workspace and are an integral part in deploying and training models.
140
+
* VMs: provide computing power for compute instances and compute clusters, which you use to run jobs.
112
141
* Load Balancer: a network load balancer is created for each compute instance and compute cluster to manage traffic even while the compute instance/cluster is stopped.
113
142
* Virtual Network: these help Azure resources communicate with one another, the internet, and other on-premises networks.
114
143
* Bandwidth: encapsulates all outbound data transfers across regions.
115
144
116
-
## Associated resources
117
-
118
-
When you create a new workspace, you're required to bring other Azure resources to store your data:
119
-
120
-
+[Azure Storage account](https://azure.microsoft.com/services/storage/): Is used as the default datastore for the workspace. Jupyter notebooks that are used with your Azure Machine Learning compute instances are stored here as well.
121
-
122
-
> [!IMPORTANT]
123
-
> By default, the storage account is a general-purpose v1 account. You can [upgrade this to general-purpose v2](../storage/common/storage-account-upgrade.md) after the workspace has been created.
124
-
> Do not enable hierarchical namespace on the storage account after upgrading to general-purpose v2.
125
-
126
-
To use an existing Azure Storage account, it can't be of type BlobStorage or a premium account (Premium_LRS and Premium_GRS). It also can't have a hierarchical namespace (used with Azure Data Lake Storage Gen2). Neither premium storage nor hierarchical namespaces are supported with the _default_ storage account of the workspace. You can use premium storage or hierarchical namespace with _non-default_ storage accounts.
127
-
128
-
+[Azure Container Registry](https://azure.microsoft.com/services/container-registry/) (ACR): When you build custom docker containers via Azure Machine Learning. For example, in the following scenarios:
129
-
*[Azure Machine Learning environments](concept-environments.md) when training and deploying models
130
-
:::moniker range="azureml-api-2"
131
-
*[AutoML](concept-automated-ml.md) when deploying
132
-
:::moniker-end
133
-
:::moniker range="azureml-api-1"
134
-
* [AutoML](./v1/concept-automated-ml-v1.md) when deploying
> Workspaces can be created without Azure Container Registry as a dependency if you do not have a need to build custom docker containers. To read container images, Azure Machine Learning also works with external container registries. Azure Container Registry is automatically provisioned when you build custom docker images. Use Azure RBAC to prevent customer docker containers from being build.
140
-
141
-
> [!NOTE]
142
-
> If your subscription setting requires adding tags to resources under it, Azure Container Registry (ACR) created by Azure Machine Learning will fail, since we cannot set tags to ACR.
143
-
144
-
+[Azure Application Insights](https://azure.microsoft.com/services/application-insights/): Stores monitoring and diagnostics information.
145
-
:::moniker range="azureml-api-2"
146
-
For more information, see [Monitor online endpoints](how-to-monitor-online-endpoints.md).
147
-
:::moniker-end
148
-
149
-
> [!NOTE]
150
-
> You can delete the Application Insights instance after cluster creation if you want. Deleting it limits the information gathered from the workspace, and may make it more difficult to troubleshoot problems. __If you delete the Application Insights instance created by the workspace, you cannot re-create it without deleting and recreating the workspace__.
151
-
152
-
+[Azure Key Vault](https://azure.microsoft.com/services/key-vault/): Stores secrets that are used by compute targets and other sensitive information that's needed by the workspace.
153
-
154
-
> [!NOTE]
155
-
> You can instead use existing Azure resource instances when you create the workspace with the [Python SDK](how-to-manage-workspace.md?tabs=python#create-a-workspace) or the Azure Machine Learning CLI [using an ARM template](how-to-create-workspace-template.md).
156
-
157
145
## Next steps
158
146
159
147
To learn more about planning a workspace for your organization's requirements, see [Organize and set up Azure Machine Learning](/azure/cloud-adoption-framework/ready/azure-best-practices/ai-machine-learning-resource-organization).
0 commit comments