You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-compute-instance.md
+12-10Lines changed: 12 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,8 @@ ms.custom: event-tier1-build-2022
9
9
ms.topic: conceptual
10
10
ms.author: sgilley
11
11
author: sdgilley
12
-
ms.date: 09/22/2021
12
+
ms.reviewer: sgilley
13
+
ms.date: 10/19/2022
13
14
#Customer intent: As a data scientist, I want to know what a compute instance is and how to use it for Azure Machine Learning.
14
15
---
15
16
@@ -89,7 +90,7 @@ Python packages are all installed in the **Python 3.8 - AzureML** environment. C
89
90
90
91
## Accessing files
91
92
92
-
Notebooks and R scripts are stored in the default storage account of your workspace in Azure file share. These files are located under your “User files” directory. This storage makes it easy to share notebooks between compute instances. The storage account also keeps your notebooks safely preserved when you stop or delete a compute instance.
93
+
Notebooks and Python scripts are stored in the default storage account of your workspace in Azure file share. These files are located under your “User files” directory. This storage makes it easy to share notebooks between compute instances. The storage account also keeps your notebooks safely preserved when you stop or delete a compute instance.
93
94
94
95
The Azure file share account of your workspace is mounted as a drive on the compute instance. This drive is the default working directory for Jupyter, Jupyter Labs, and RStudio. This means that the notebooks and other files you create in Jupyter, JupyterLab, or RStudio are automatically stored on the file share and available to use in other compute instances as well.
95
96
@@ -101,17 +102,18 @@ Writing small files can be slower on network drives than writing to the compute
101
102
102
103
Do not store training data on the notebooks file share. You can use the `/tmp` directory on the compute instance for your temporary data. However, do not write very large files of data on the OS disk of the compute instance. OS disk on compute instance has 128 GB capacity. You can also store temporary training data on temporary disk mounted on /mnt. Temporary disk size is configurable based on the VM size chosen and can store larger amounts of data if a higher size VM is chosen. You can also mount [datastores and datasets](v1/concept-azure-machine-learning-architecture.md#datasets-and-datastores). Any software packages you install are saved on the OS disk of compute instance. Please note customer managed key encryption is currently not supported for OS disk. The OS disk for compute instance is encrypted with Microsoft-managed keys.
103
104
104
-
### Create
105
+
## Create
106
+
107
+
Follow the steps in the [Quickstart: Create workspace resources you need to get started with Azure Machine Learning](quickstart-create-resources.md) to create a basic compute instance.
108
+
109
+
For more options, see [create a new compute instance](how-to-create-manage-compute-instance.md?tabs=azure-studio#create).
105
110
106
111
As an administrator, you can **[create a compute instance for others in the workspace (preview)](how-to-create-manage-compute-instance.md#create-on-behalf-of-preview)**.
107
112
108
113
You can also **[use a setup script (preview)](how-to-customize-compute-instance.md)** for an automated way to customize and configure the compute instance.
109
114
110
-
To create a compute instance for yourself, use your workspace in Azure Machine Learning studio, [create a new compute instance](how-to-create-manage-compute-instance.md?tabs=azure-studio#create) from either the **Compute** section or in the **Notebooks** section when you are ready to run one of your notebooks.
111
-
112
-
You can also create an instance
113
-
* Directly from the [integrated notebooks experience](tutorial-train-deploy-notebook.md#azure)
114
-
* In Azure portal
115
+
Other ways to create a compute instance:
116
+
* Directly from the integrated notebooks experience.
115
117
* From Azure Resource Manager template. For an example template, see the [create an Azure Machine Learning compute instance template](https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.machinelearningservices/machine-learning-compute-create-computeinstance).
116
118
* With [Azure Machine Learning SDK](how-to-create-manage-compute-instance.md?tabs=python#create)
117
119
* From the [CLI extension for Azure Machine Learning](how-to-create-manage-compute-instance.md?tabs=azure-cli#create)
@@ -123,7 +125,7 @@ Compute instance comes with P10 OS disk. Temp disk type depends on the VM size c
123
125
124
126
## Compute target
125
127
126
-
Compute instances can be used as a [training compute target](concept-compute-target.md#train) similar to Azure Machine Learning [compute training clusters](how-to-create-attach-compute-cluster.md). But a compute instance has only a single node, while a compute cluster can have more nodes.
128
+
Compute instances can be used as a [training compute target](concept-compute-target.md#training-compute-targets) similar to Azure Machine Learning [compute training clusters](how-to-create-attach-compute-cluster.md). But a compute instance has only a single node, while a compute cluster can have more nodes.
127
129
128
130
A compute instance:
129
131
@@ -139,5 +141,5 @@ You can use compute instance as a local inferencing deployment target for test/d
139
141
140
142
## Next steps
141
143
142
-
*[Create and manage a compute instance](how-to-create-manage-compute-instance.md)
144
+
*[Quickstart: Create workspace resources you need to get started with Azure Machine Learning](quickstart-create-resources.md).
143
145
*[Tutorial: Train your first ML model](tutorial-1st-experiment-sdk-train.md) shows how to use a compute instance with an integrated notebook.
#Customer intent: As a data scientist, I want to understand what a compute target is and why I need it.
14
15
---
@@ -20,27 +21,26 @@ A *compute target* is a designated compute resource or environment where you run
20
21
In a typical model development lifecycle, you might:
21
22
22
23
1. Start by developing and experimenting on a small amount of data. At this stage, use your local environment, such as a local computer or cloud-based virtual machine (VM), as your compute target.
23
-
1. Scale up to larger data, or do [distributed training](how-to-train-distributed-gpu.md) by using one of these [training compute targets](#train).
24
-
1. After your model is ready, deploy it to a web hosting environment with one of these [deployment compute targets](#deploy).
24
+
1. Scale up to larger data, or do [distributed training](how-to-train-distributed-gpu.md) by using one of these [training compute targets](#training-compute-targets).
25
+
1. After your model is ready, deploy it to a web hosting environment with one of these [deployment compute targets](#compute-targets-for-inference).
25
26
26
27
The compute resources you use for your compute targets are attached to a [workspace](concept-workspace.md). Compute resources other than the local machine are shared by users of the workspace.
27
28
28
-
## <aname="train"></a> Training compute targets
29
+
## Training compute targets
29
30
30
31
Azure Machine Learning has varying support across different compute targets. A typical model development lifecycle starts with development or experimentation on a small amount of data. At this stage, use a local environment like your local computer or a cloud-based VM. As you scale up your training on larger datasets or perform [distributed training](how-to-train-distributed-gpu.md), use Azure Machine Learning compute to create a single- or multi-node cluster that autoscales each time you submit a job. You can also attach your own compute resource, although support for different scenarios might vary.
## <aname="deploy"></a> Compute targets for inference
36
+
## Compute targets for inference
36
37
37
38
When performing inference, Azure Machine Learning creates a Docker container that hosts the model and associated resources needed to use it. This container is then used in a compute target.
Learn [where and how to deploy your model to a compute target](how-to-deploy-managed-online-endpoints.md).
42
43
43
-
<aname="amlcompute"></a>
44
44
## Azure Machine Learning compute (managed)
45
45
46
46
A managed compute resource is created and managed by Azure Machine Learning. This compute is optimized for machine learning workloads. Azure Machine Learning compute clusters and [compute instances](concept-compute-instance.md) are the only managed computes.
The compute autoscales down to zero nodes when it isn't used. Dedicated VMs are created to run your jobs as needed.
80
+
81
+
The fastest way to create a compute cluster is to follow the [Quickstart: Create workspace resources you need to get started with Azure Machine Learning](quickstart-create-resources.md).
82
+
83
+
Or use the following examples to create a compute cluster with more options:
Learn how to create and manage a [compute instance](concept-compute-instance.md) in your Azure Machine Learning workspace.
24
+
Learn how to create and manage a [compute instance](concept-compute-instance.md) in your Azure Machine Learning workspace.
25
25
26
-
Use a compute instance as your fully configured and managed development environment in the cloud. For development and testing, you can also use the instance as a [training compute target](concept-compute-target.md#train). A compute instance can run multiple jobs in parallel and has a job queue. As a development environment, a compute instance can't be shared with other users in your workspace.
26
+
Use a compute instance as your fully configured and managed development environment in the cloud. For development and testing, you can also use the instance as a [training compute target](concept-compute-target.md#training-compute-targets). A compute instance can run multiple jobs in parallel and has a job queue. As a development environment, a compute instance can't be shared with other users in your workspace.
27
27
28
28
In this article, you learn how to:
29
29
@@ -62,7 +62,9 @@ Creating a compute instance is a one time process for your workspace. You can re
62
62
63
63
The dedicated cores per region per VM family quota and total regional quota, which applies to compute instance creation, is unified and shared with Azure Machine Learning training compute cluster quota. Stopping the compute instance doesn't release quota to ensure you'll be able to restart the compute instance. It isn't possible to change the virtual machine size of compute instance once it's created.
64
64
65
-
The following example demonstrates how to create a compute instance:
65
+
The fastest way to create a compute instance is to follow the [Quickstart: Create workspace resources you need to get started with Azure Machine Learning](quickstart-create-resources.md).
66
+
67
+
Or use the following examples to create a compute instance with more options:
66
68
67
69
# [Python SDK](#tab/python)
68
70
@@ -240,7 +242,6 @@ You can also create your own custom Azure policy. For example, if the below poli
240
242
}
241
243
```
242
244
243
-
244
245
## Create on behalf of (preview)
245
246
246
247
As an administrator, you can create a compute instance on behalf of a data scientist and assign the instance to them with:
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-customize-compute-instance.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,7 +22,7 @@ ms.date: 05/04/2022
22
22
23
23
Use a setup script for an automated way to customize and configure a compute instance at provisioning time.
24
24
25
-
Use a compute instance as your fully configured and managed development environment in the cloud. For development and testing, you can also use the instance as a [training compute target](concept-compute-target.md#train) or for an [inference target](concept-compute-target.md#deploy). A compute instance can run multiple jobs in parallel and has a job queue. As a development environment, a compute instance can't be shared with other users in your workspace.
25
+
Use a compute instance as your fully configured and managed development environment in the cloud. For development and testing, you can also use the instance as a [training compute target](concept-compute-target.md#training-compute-targets) or for an [inference target](concept-compute-target.md#compute-targets-for-inference). A compute instance can run multiple jobs in parallel and has a job queue. As a development environment, a compute instance can't be shared with other users in your workspace.
26
26
27
27
As an administrator, you can write a customization script to be used to provision all compute instances in the workspace according to your requirements.
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-manage-resources-vscode.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -135,7 +135,7 @@ Alternatively, use the `> Azure ML: Create Dataset` command in the command palet
135
135
1. Expand your workspace node.
136
136
1. Expand the **Datasets** node.
137
137
1. Right-click the dataset you want to:
138
-
-**View Dataset Properties**. Lets you view metadata associated with a specific dataset. If you have multiple version of a dataset, you can choose to only view the dataset properties of a specific version by expanding the dataset node and performing the same steps described in this section on the version of interest.
138
+
-**View Dataset Properties**. Lets you view metadata associated with a specific dataset. If you have multiple versions of a dataset, you can choose to only view the dataset properties of a specific version by expanding the dataset node and performing the same steps described in this section on the version of interest.
139
139
-**Preview dataset**. View your dataset directly in the VS Code Data Viewer. Note that this option is only available for tabular datasets.
140
140
-**Unregister dataset**. Removes a dataset and all versions of it from your workspace.
141
141
@@ -267,7 +267,7 @@ Alternatively, use the `Azure ML: Delete Compute instance` command in the comman
267
267
268
268
## Compute clusters
269
269
270
-
For more information, see [training compute targets](concept-compute-target.md#train).
270
+
For more information, see [training compute targets](concept-compute-target.md#training-compute-targets).
271
271
272
272
### Create compute cluster
273
273
@@ -300,7 +300,7 @@ Alternatively, use the `> Azure ML: Remove Compute` command in the command palet
300
300
301
301
## Inference Clusters
302
302
303
-
For more information, see [compute targets for inference](concept-compute-target.md#deploy).
303
+
For more information, see [compute targets for inference](concept-compute-target.md#compute-targets-for-inference).
***Compute clusters**: Compute clusters are a cluster of VMs with multi-node scaling capabilities. Compute clusters are better suited for compute targets for large jobs and production. The cluster scales up automatically when a job is submitted. Use as a training compute target or for dev/test deployment.
60
60
61
-
For more information about training compute targets, see [Training compute targets](../concept-compute-target.md#train). For more information about deployment compute targets, see [Deployment targets](../concept-compute-target.md#deploy).
61
+
For more information about training compute targets, see [Training compute targets](../concept-compute-target.md#training-compute-targets). For more information about deployment compute targets, see [Deployment targets](../concept-compute-target.md#compute-targets-for-inference).
Copy file name to clipboardExpand all lines: articles/machine-learning/v1/how-to-configure-auto-train-v1.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -141,7 +141,7 @@ Next determine where the model will be trained. An automated ML training experim
141
141
142
142
***Choose a local compute**: If your scenario is about initial explorations or demos using small data and short trains (i.e. seconds or a couple of minutes per child run), training on your local computer might be a better choice. There is no setup time, the infrastructure resources (your PC or VM) are directly available. See [this notebook](https://github.com/Azure/azureml-examples/blob/main/v1/python-sdk/tutorials/automl-with-azureml/local-run-classification-credit-card-fraud/auto-ml-classification-credit-card-fraud-local.ipynb) for a local compute example.
143
143
144
-
***Choose a remote ML compute cluster**: If you are training with larger datasets like in production training creating models which need longer trains, remote compute will provide much better end-to-end time performance because `AutoML` will parallelize trains across the cluster's nodes. On a remote compute, the start-up time for the internal infrastructure will add around 1.5 minutes per child run, plus additional minutes for the cluster infrastructure if the VMs are not yet up and running.[Azure Machine Learning Managed Compute](../concept-compute-target.md#amlcompute) is a managed service that enables the ability to train machine learning models on clusters of Azure virtual machines. Compute instance is also supported as a compute target.
144
+
***Choose a remote ML compute cluster**: If you are training with larger datasets like in production training creating models which need longer trains, remote compute will provide much better end-to-end time performance because `AutoML` will parallelize trains across the cluster's nodes. On a remote compute, the start-up time for the internal infrastructure will add around 1.5 minutes per child run, plus additional minutes for the cluster infrastructure if the VMs are not yet up and running.[Azure Machine Learning Managed Compute](../concept-compute-target.md#azure-machine-learning-compute-managed) is a managed service that enables the ability to train machine learning models on clusters of Azure virtual machines. Compute instance is also supported as a compute target.
145
145
146
146
* An **Azure Databricks cluster** in your Azure subscription. You can find more details in [Set up an Azure Databricks cluster for automated ML](../how-to-configure-databricks-automl-environment.md). See this [GitHub site](https://github.com/Azure/azureml-examples/tree/main/v1/python-sdk/tutorials/automl-with-databricks) for examples of notebooks with Azure Databricks.
In Azure Machine Learning, the term __compute__ (or __compute target__) refers to the machines or clusters that do the computational steps in your machine learning pipeline. See [compute targets for model training](../concept-compute-target.md#train) for a full list of compute targets and [Create compute targets](../how-to-create-attach-compute-studio.md) for how to create and attach them to your workspace. The process for creating and or attaching a compute target is the same whether you're training a model or running a pipeline step. After you create and attach your compute target, use the `ComputeTarget` object in your [pipeline step](#steps).
112
+
In Azure Machine Learning, the term __compute__ (or __compute target__) refers to the machines or clusters that do the computational steps in your machine learning pipeline. See [compute targets for model training](../concept-compute-target.md#training-compute-targets) for a full list of compute targets and [Create compute targets](../how-to-create-attach-compute-studio.md) for how to create and attach them to your workspace. The process for creating and or attaching a compute target is the same whether you're training a model or running a pipeline step. After you create and attach your compute target, use the `ComputeTarget` object in your [pipeline step](#steps).
113
113
114
114
> [!IMPORTANT]
115
115
> Performing management operations on compute targets isn't supported from inside remote jobs. Since machine learning pipelines are submitted as a remote job, do not use management operations on compute targets from inside the pipeline.
0 commit comments