Skip to content

Commit 73c54c6

Browse files
Merge pull request #215129 from sdgilley/sdg-content-freshness
freshness & add links to quickstart
2 parents c06e65c + 2a728f6 commit 73c54c6

11 files changed

+39
-32
lines changed

articles/machine-learning/concept-compute-instance.md

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@ ms.custom: event-tier1-build-2022
99
ms.topic: conceptual
1010
ms.author: sgilley
1111
author: sdgilley
12-
ms.date: 09/22/2021
12+
ms.reviewer: sgilley
13+
ms.date: 10/19/2022
1314
#Customer intent: As a data scientist, I want to know what a compute instance is and how to use it for Azure Machine Learning.
1415
---
1516

@@ -89,7 +90,7 @@ Python packages are all installed in the **Python 3.8 - AzureML** environment. C
8990

9091
## Accessing files
9192

92-
Notebooks and R scripts are stored in the default storage account of your workspace in Azure file share. These files are located under your “User files” directory. This storage makes it easy to share notebooks between compute instances. The storage account also keeps your notebooks safely preserved when you stop or delete a compute instance.
93+
Notebooks and Python scripts are stored in the default storage account of your workspace in Azure file share. These files are located under your “User files” directory. This storage makes it easy to share notebooks between compute instances. The storage account also keeps your notebooks safely preserved when you stop or delete a compute instance.
9394

9495
The Azure file share account of your workspace is mounted as a drive on the compute instance. This drive is the default working directory for Jupyter, Jupyter Labs, and RStudio. This means that the notebooks and other files you create in Jupyter, JupyterLab, or RStudio are automatically stored on the file share and available to use in other compute instances as well.
9596

@@ -101,17 +102,18 @@ Writing small files can be slower on network drives than writing to the compute
101102

102103
Do not store training data on the notebooks file share. You can use the `/tmp` directory on the compute instance for your temporary data. However, do not write very large files of data on the OS disk of the compute instance. OS disk on compute instance has 128 GB capacity. You can also store temporary training data on temporary disk mounted on /mnt. Temporary disk size is configurable based on the VM size chosen and can store larger amounts of data if a higher size VM is chosen. You can also mount [datastores and datasets](v1/concept-azure-machine-learning-architecture.md#datasets-and-datastores). Any software packages you install are saved on the OS disk of compute instance. Please note customer managed key encryption is currently not supported for OS disk. The OS disk for compute instance is encrypted with Microsoft-managed keys.
103104

104-
### Create
105+
## Create
106+
107+
Follow the steps in the [Quickstart: Create workspace resources you need to get started with Azure Machine Learning](quickstart-create-resources.md) to create a basic compute instance.
108+
109+
For more options, see [create a new compute instance](how-to-create-manage-compute-instance.md?tabs=azure-studio#create).
105110

106111
As an administrator, you can **[create a compute instance for others in the workspace (preview)](how-to-create-manage-compute-instance.md#create-on-behalf-of-preview)**.
107112

108113
You can also **[use a setup script (preview)](how-to-customize-compute-instance.md)** for an automated way to customize and configure the compute instance.
109114

110-
To create a compute instance for yourself, use your workspace in Azure Machine Learning studio, [create a new compute instance](how-to-create-manage-compute-instance.md?tabs=azure-studio#create) from either the **Compute** section or in the **Notebooks** section when you are ready to run one of your notebooks.
111-
112-
You can also create an instance
113-
* Directly from the [integrated notebooks experience](tutorial-train-deploy-notebook.md#azure)
114-
* In Azure portal
115+
Other ways to create a compute instance:
116+
* Directly from the integrated notebooks experience.
115117
* From Azure Resource Manager template. For an example template, see the [create an Azure Machine Learning compute instance template](https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.machinelearningservices/machine-learning-compute-create-computeinstance).
116118
* With [Azure Machine Learning SDK](how-to-create-manage-compute-instance.md?tabs=python#create)
117119
* From the [CLI extension for Azure Machine Learning](how-to-create-manage-compute-instance.md?tabs=azure-cli#create)
@@ -123,7 +125,7 @@ Compute instance comes with P10 OS disk. Temp disk type depends on the VM size c
123125

124126
## Compute target
125127

126-
Compute instances can be used as a [training compute target](concept-compute-target.md#train) similar to Azure Machine Learning [compute training clusters](how-to-create-attach-compute-cluster.md). But a compute instance has only a single node, while a compute cluster can have more nodes.
128+
Compute instances can be used as a [training compute target](concept-compute-target.md#training-compute-targets) similar to Azure Machine Learning [compute training clusters](how-to-create-attach-compute-cluster.md). But a compute instance has only a single node, while a compute cluster can have more nodes.
127129

128130
A compute instance:
129131

@@ -139,5 +141,5 @@ You can use compute instance as a local inferencing deployment target for test/d
139141
140142
## Next steps
141143

142-
* [Create and manage a compute instance](how-to-create-manage-compute-instance.md)
144+
* [Quickstart: Create workspace resources you need to get started with Azure Machine Learning](quickstart-create-resources.md).
143145
* [Tutorial: Train your first ML model](tutorial-1st-experiment-sdk-train.md) shows how to use a compute instance with an integrated notebook.

articles/machine-learning/concept-compute-target.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@ ms.subservice: core
88
ms.topic: conceptual
99
ms.author: sgilley
1010
author: sdgilley
11-
ms.date: 10/21/2021
11+
ms.reviewer: sgilley
12+
ms.date: 10/19/2022
1213
ms.custom: ignite-fall-2021, event-tier1-build-2022, cliv2
1314
#Customer intent: As a data scientist, I want to understand what a compute target is and why I need it.
1415
---
@@ -20,27 +21,26 @@ A *compute target* is a designated compute resource or environment where you run
2021
In a typical model development lifecycle, you might:
2122

2223
1. Start by developing and experimenting on a small amount of data. At this stage, use your local environment, such as a local computer or cloud-based virtual machine (VM), as your compute target.
23-
1. Scale up to larger data, or do [distributed training](how-to-train-distributed-gpu.md) by using one of these [training compute targets](#train).
24-
1. After your model is ready, deploy it to a web hosting environment with one of these [deployment compute targets](#deploy).
24+
1. Scale up to larger data, or do [distributed training](how-to-train-distributed-gpu.md) by using one of these [training compute targets](#training-compute-targets).
25+
1. After your model is ready, deploy it to a web hosting environment with one of these [deployment compute targets](#compute-targets-for-inference).
2526

2627
The compute resources you use for your compute targets are attached to a [workspace](concept-workspace.md). Compute resources other than the local machine are shared by users of the workspace.
2728

28-
## <a name="train"></a> Training compute targets
29+
## Training compute targets
2930

3031
Azure Machine Learning has varying support across different compute targets. A typical model development lifecycle starts with development or experimentation on a small amount of data. At this stage, use a local environment like your local computer or a cloud-based VM. As you scale up your training on larger datasets or perform [distributed training](how-to-train-distributed-gpu.md), use Azure Machine Learning compute to create a single- or multi-node cluster that autoscales each time you submit a job. You can also attach your own compute resource, although support for different scenarios might vary.
3132

3233
[!INCLUDE [aml-compute-target-train](../../includes/aml-compute-target-train.md)]
3334

3435

35-
## <a name="deploy"></a> Compute targets for inference
36+
## Compute targets for inference
3637

3738
When performing inference, Azure Machine Learning creates a Docker container that hosts the model and associated resources needed to use it. This container is then used in a compute target.
3839

3940
[!INCLUDE [aml-deploy-target](../../includes/aml-compute-target-deploy.md)]
4041

4142
Learn [where and how to deploy your model to a compute target](how-to-deploy-managed-online-endpoints.md).
4243

43-
<a name="amlcompute"></a>
4444
## Azure Machine Learning compute (managed)
4545

4646
A managed compute resource is created and managed by Azure Machine Learning. This compute is optimized for machine learning workloads. Azure Machine Learning compute clusters and [compute instances](concept-compute-instance.md) are the only managed computes.

articles/machine-learning/how-to-create-attach-compute-cluster.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ ms.custom: devx-track-azurecli, cliv2, sdkv1, event-tier1-build-2022
1010
ms.author: sgilley
1111
author: sdgilley
1212
ms.reviewer: sgilley
13-
ms.date: 09/21/2022
13+
ms.date: 10/19/2022
1414
---
1515

1616
# Create an Azure Machine Learning compute cluster
@@ -77,6 +77,10 @@ The dedicated cores per region per VM family quota and total regional quota, whi
7777
[!INCLUDE [min-nodes-note](../../includes/machine-learning-min-nodes.md)]
7878

7979
The compute autoscales down to zero nodes when it isn't used. Dedicated VMs are created to run your jobs as needed.
80+
81+
The fastest way to create a compute cluster is to follow the [Quickstart: Create workspace resources you need to get started with Azure Machine Learning](quickstart-create-resources.md).
82+
83+
Or use the following examples to create a compute cluster with more options:
8084

8185
# [Python SDK](#tab/python)
8286

articles/machine-learning/how-to-create-manage-compute-instance.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ ms.topic: how-to
1010
author: swatig007
1111
ms.author: swatig
1212
ms.reviewer: sgilley
13-
ms.date: 09/21/2022
13+
ms.date: 10/19/2022
1414
---
1515

1616
# Create and manage an Azure Machine Learning compute instance
@@ -21,9 +21,9 @@ ms.date: 09/21/2022
2121
> * [v1](v1/how-to-create-manage-compute-instance.md)
2222
> * [v2 (current version)](how-to-create-manage-compute-instance.md)
2323
24-
Learn how to create and manage a [compute instance](concept-compute-instance.md) in your Azure Machine Learning workspace.
24+
Learn how to create and manage a [compute instance](concept-compute-instance.md) in your Azure Machine Learning workspace.
2525

26-
Use a compute instance as your fully configured and managed development environment in the cloud. For development and testing, you can also use the instance as a [training compute target](concept-compute-target.md#train). A compute instance can run multiple jobs in parallel and has a job queue. As a development environment, a compute instance can't be shared with other users in your workspace.
26+
Use a compute instance as your fully configured and managed development environment in the cloud. For development and testing, you can also use the instance as a [training compute target](concept-compute-target.md#training-compute-targets). A compute instance can run multiple jobs in parallel and has a job queue. As a development environment, a compute instance can't be shared with other users in your workspace.
2727

2828
In this article, you learn how to:
2929

@@ -62,7 +62,9 @@ Creating a compute instance is a one time process for your workspace. You can re
6262

6363
The dedicated cores per region per VM family quota and total regional quota, which applies to compute instance creation, is unified and shared with Azure Machine Learning training compute cluster quota. Stopping the compute instance doesn't release quota to ensure you'll be able to restart the compute instance. It isn't possible to change the virtual machine size of compute instance once it's created.
6464

65-
The following example demonstrates how to create a compute instance:
65+
The fastest way to create a compute instance is to follow the [Quickstart: Create workspace resources you need to get started with Azure Machine Learning](quickstart-create-resources.md).
66+
67+
Or use the following examples to create a compute instance with more options:
6668

6769
# [Python SDK](#tab/python)
6870

@@ -240,7 +242,6 @@ You can also create your own custom Azure policy. For example, if the below poli
240242
}
241243
```
242244

243-
244245
## Create on behalf of (preview)
245246

246247
As an administrator, you can create a compute instance on behalf of a data scientist and assign the instance to them with:

articles/machine-learning/how-to-customize-compute-instance.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ ms.date: 05/04/2022
2222
2323
Use a setup script for an automated way to customize and configure a compute instance at provisioning time.
2424

25-
Use a compute instance as your fully configured and managed development environment in the cloud. For development and testing, you can also use the instance as a [training compute target](concept-compute-target.md#train) or for an [inference target](concept-compute-target.md#deploy). A compute instance can run multiple jobs in parallel and has a job queue. As a development environment, a compute instance can't be shared with other users in your workspace.
25+
Use a compute instance as your fully configured and managed development environment in the cloud. For development and testing, you can also use the instance as a [training compute target](concept-compute-target.md#training-compute-targets) or for an [inference target](concept-compute-target.md#compute-targets-for-inference). A compute instance can run multiple jobs in parallel and has a job queue. As a development environment, a compute instance can't be shared with other users in your workspace.
2626

2727
As an administrator, you can write a customization script to be used to provision all compute instances in the workspace according to your requirements.
2828

articles/machine-learning/how-to-manage-resources-vscode.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -135,7 +135,7 @@ Alternatively, use the `> Azure ML: Create Dataset` command in the command palet
135135
1. Expand your workspace node.
136136
1. Expand the **Datasets** node.
137137
1. Right-click the dataset you want to:
138-
- **View Dataset Properties**. Lets you view metadata associated with a specific dataset. If you have multiple version of a dataset, you can choose to only view the dataset properties of a specific version by expanding the dataset node and performing the same steps described in this section on the version of interest.
138+
- **View Dataset Properties**. Lets you view metadata associated with a specific dataset. If you have multiple versions of a dataset, you can choose to only view the dataset properties of a specific version by expanding the dataset node and performing the same steps described in this section on the version of interest.
139139
- **Preview dataset**. View your dataset directly in the VS Code Data Viewer. Note that this option is only available for tabular datasets.
140140
- **Unregister dataset**. Removes a dataset and all versions of it from your workspace.
141141

@@ -267,7 +267,7 @@ Alternatively, use the `Azure ML: Delete Compute instance` command in the comman
267267

268268
## Compute clusters
269269

270-
For more information, see [training compute targets](concept-compute-target.md#train).
270+
For more information, see [training compute targets](concept-compute-target.md#training-compute-targets).
271271

272272
### Create compute cluster
273273

@@ -300,7 +300,7 @@ Alternatively, use the `> Azure ML: Remove Compute` command in the command palet
300300

301301
## Inference Clusters
302302

303-
For more information, see [compute targets for inference](concept-compute-target.md#deploy).
303+
For more information, see [compute targets for inference](concept-compute-target.md#compute-targets-for-inference).
304304

305305
### Manage inference clusters
306306

articles/machine-learning/v1/concept-azure-machine-learning-architecture.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ Azure Machine Learning introduces two fully managed cloud-based virtual machines
5858

5959
* **Compute clusters**: Compute clusters are a cluster of VMs with multi-node scaling capabilities. Compute clusters are better suited for compute targets for large jobs and production. The cluster scales up automatically when a job is submitted. Use as a training compute target or for dev/test deployment.
6060

61-
For more information about training compute targets, see [Training compute targets](../concept-compute-target.md#train). For more information about deployment compute targets, see [Deployment targets](../concept-compute-target.md#deploy).
61+
For more information about training compute targets, see [Training compute targets](../concept-compute-target.md#training-compute-targets). For more information about deployment compute targets, see [Deployment targets](../concept-compute-target.md#compute-targets-for-inference).
6262

6363
## Datasets and datastores
6464

articles/machine-learning/v1/how-to-configure-auto-train-v1.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,7 @@ Next determine where the model will be trained. An automated ML training experim
141141

142142
* **Choose a local compute**: If your scenario is about initial explorations or demos using small data and short trains (i.e. seconds or a couple of minutes per child run), training on your local computer might be a better choice. There is no setup time, the infrastructure resources (your PC or VM) are directly available. See [this notebook](https://github.com/Azure/azureml-examples/blob/main/v1/python-sdk/tutorials/automl-with-azureml/local-run-classification-credit-card-fraud/auto-ml-classification-credit-card-fraud-local.ipynb) for a local compute example.
143143

144-
* **Choose a remote ML compute cluster**: If you are training with larger datasets like in production training creating models which need longer trains, remote compute will provide much better end-to-end time performance because `AutoML` will parallelize trains across the cluster's nodes. On a remote compute, the start-up time for the internal infrastructure will add around 1.5 minutes per child run, plus additional minutes for the cluster infrastructure if the VMs are not yet up and running.[Azure Machine Learning Managed Compute](../concept-compute-target.md#amlcompute) is a managed service that enables the ability to train machine learning models on clusters of Azure virtual machines. Compute instance is also supported as a compute target.
144+
* **Choose a remote ML compute cluster**: If you are training with larger datasets like in production training creating models which need longer trains, remote compute will provide much better end-to-end time performance because `AutoML` will parallelize trains across the cluster's nodes. On a remote compute, the start-up time for the internal infrastructure will add around 1.5 minutes per child run, plus additional minutes for the cluster infrastructure if the VMs are not yet up and running.[Azure Machine Learning Managed Compute](../concept-compute-target.md#azure-machine-learning-compute-managed) is a managed service that enables the ability to train machine learning models on clusters of Azure virtual machines. Compute instance is also supported as a compute target.
145145

146146
* An **Azure Databricks cluster** in your Azure subscription. You can find more details in [Set up an Azure Databricks cluster for automated ML](../how-to-configure-databricks-automl-environment.md). See this [GitHub site](https://github.com/Azure/azureml-examples/tree/main/v1/python-sdk/tutorials/automl-with-databricks) for examples of notebooks with Azure Databricks.
147147

articles/machine-learning/v1/how-to-create-machine-learning-pipelines.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ output_data_dataset = output_data1.register_on_complete(name = 'prepared_output_
109109
## Set up a compute target
110110

111111

112-
In Azure Machine Learning, the term __compute__ (or __compute target__) refers to the machines or clusters that do the computational steps in your machine learning pipeline. See [compute targets for model training](../concept-compute-target.md#train) for a full list of compute targets and [Create compute targets](../how-to-create-attach-compute-studio.md) for how to create and attach them to your workspace. The process for creating and or attaching a compute target is the same whether you're training a model or running a pipeline step. After you create and attach your compute target, use the `ComputeTarget` object in your [pipeline step](#steps).
112+
In Azure Machine Learning, the term __compute__ (or __compute target__) refers to the machines or clusters that do the computational steps in your machine learning pipeline. See [compute targets for model training](../concept-compute-target.md#training-compute-targets) for a full list of compute targets and [Create compute targets](../how-to-create-attach-compute-studio.md) for how to create and attach them to your workspace. The process for creating and or attaching a compute target is the same whether you're training a model or running a pipeline step. After you create and attach your compute target, use the `ComputeTarget` object in your [pipeline step](#steps).
113113

114114
> [!IMPORTANT]
115115
> Performing management operations on compute targets isn't supported from inside remote jobs. Since machine learning pipelines are submitted as a remote job, do not use management operations on compute targets from inside the pipeline.

0 commit comments

Comments
 (0)