You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/data-science-virtual-machine/dsvm-pools.md
+22-17Lines changed: 22 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,46 +9,51 @@ ms.service: data-science-vm
9
9
author: vijetajo
10
10
ms.author: vijetaj
11
11
ms.topic: conceptual
12
-
ms.date: 12/10/2018
12
+
ms.reviewer: franksolomon
13
+
ms.date: 04/11/2024
13
14
---
14
15
15
16
# Create a shared pool of Data Science Virtual Machines
16
17
17
-
In this article, you'll learn how to create a shared pool of Data Science Virtual Machines (DSVMs) for a team. The benefits of using a shared pool include better resource utilization, easier sharing and collaboration, and more effective management of DSVM resources.
18
+
In this article, you'll learn how to create a shared pool of Data Science Virtual Machines (DSVMs) for a team. Use of a shared pool offers important advantages:
18
19
19
-
You can use many methods and technologies to create a pool of DSVMs. This article focuses on pools for interactive virtual machines (VMs). An alternative managed compute infrastructure is Azure Machine Learning Compute. For more information, see [Create compute cluster](../how-to-create-attach-compute-cluster.md).
20
+
- Better resource utilization
21
+
- Easier sharing and collaboration
22
+
- More effective management of DSVM resources
23
+
24
+
You can use many methods and technologies to create a pool of DSVMs. This article focuses on pools for interactive virtual machines (VMs). An alternative managed compute infrastructure involves Azure Machine Learning Compute. For more information, visit [Create compute cluster](../how-to-create-attach-compute-cluster.md).
20
25
21
26
## Interactive VM pool
22
27
23
-
A pool of interactive VMs that are shared by the whole AI/data science team allows users to log in to an available instance of the DSVM instead of having a dedicated instance for each set of users. This setup enables better availability and more effective utilization of resources.
28
+
A pool of interactive VM, shared by an entire AI/data science team, offers users a way to sign in to an available DSVM instance, instead of having a dedicated instance for each set of users. This approach provides better availability and more effective resource utilization.
24
29
25
-
You use [Azure virtual machine scale sets](../../virtual-machine-scale-sets/index.yml) technology to create an interactive VM pool. You can use scale sets to create and manage a group of identical, load-balanced, and autoscaling VMs.
30
+
Use [Azure virtual machine scale sets](../../virtual-machine-scale-sets/index.yml) technology to create an interactive VM pool. Use scale sets to create and manage a group of identical, load-balanced, and autoscaling VMs.
26
31
27
-
The user logs in to the main pool's IP or DNS address. The scale set automatically routes the session to an available DSVM in the scale set. Because users want a consistent and familiar environment regardless of the VM they're logging in to, all instances of the VM in the scale set mount a shared network drive, like an Azure Files share or a Network File System (NFS) share. The user's shared workspace is normally kept on the shared file store that's mounted on each of the instances.
32
+
The user logs in to the IP or DNS address of the main pool. The scale set automatically routes the session to an available DSVM in the scale set. Because users want a consistent and familiar environment, regardless of the VM they sign in to, all instances of the VM in the scale set mount a shared network drive. This is similar to an Azure Files share or a Network File System (NFS) share. The user's shared workspace is normally kept on the shared file store mounted on each of the instances.
28
33
29
-
You can find a sample Azure Resource Manager template that creates a scale set with Ubuntu DSVM instances on [GitHub](https://raw.githubusercontent.com/Azure/DataScienceVM/master/Scripts/CreateDSVM/Ubuntu/dsvm-vmss-cluster.json). You'll find a sample of the [parameter file](https://raw.githubusercontent.com/Azure/DataScienceVM/master/Scripts/CreateDSVM/Ubuntu/dsvm-vmss-cluster.parameters.json) for the Azure Resource Manager template in the same location.
34
+
You can find a sample Azure Resource Manager template that creates a scale set with Ubuntu DSVM instances on [GitHub](https://raw.githubusercontent.com/Azure/DataScienceVM/master/Scripts/CreateDSVM/Ubuntu/dsvm-vmss-cluster.json). The same location hosts a sample of the [parameter file](https://raw.githubusercontent.com/Azure/DataScienceVM/master/Scripts/CreateDSVM/Ubuntu/dsvm-vmss-cluster.parameters.json) for the Azure Resource Manager template.
30
35
31
-
You can create the scale set from the Azure Resource Manager template by specifying values for the parameter file in the Azure CLI:
36
+
Specify values for the parameter file in the Azure CLI, to create the scale set from the Azure Resource Manager template:
32
37
33
38
```azurecli-interactive
34
39
az group create --name [[NAME OF RESOURCE GROUP]] --location [[ Data center. For eg: "West US 2"]
35
40
az deployment group create --resource-group [[NAME OF RESOURCE GROUP ABOVE]] --template-uri https://raw.githubusercontent.com/Azure/DataScienceVM/master/Scripts/CreateDSVM/Ubuntu/dsvm-vmss-cluster.json --parameters @[[PARAMETER JSON FILE]]
36
41
```
37
42
38
-
The preceding commands assume you have:
43
+
Those commands assume you have:
39
44
40
-
* A copy of the parameter file with the values specified for your instance of the scale set.
41
-
* The number of VM instances.
42
-
* Pointers to the Azure Files share.
43
-
* Credentials for the storage account that will be mounted on each VM.
45
+
* A copy of the parameter file with the values specified for your instance of the scale set
46
+
* The number of VM instances
47
+
* Pointers to the Azure Files share
48
+
* Credentials for the storage account that will be mounted on each VM
44
49
45
-
The parameter file is referenced locally in the commands. You can also pass parameters inline or prompt for them in your script.
50
+
The commands locally reference the parameter file. You can also pass parameters inline, or prompt for them in your script.
46
51
47
-
The preceding template enables the SSH and the JupyterHub port from the front-end scale set to the back-end pool of Ubuntu DSVMs. As a user, you log in to the VM on a Secure Shell (SSH) or on JupyterHub in the normal way. Because the VM instances can be scaled up or down dynamically, any state must be saved in the mounted Azure Files share. You can use the same approach to create a pool of Windows DSVMs.
52
+
The preceding template enables the SSH and the JupyterHub port from the front-end scale set to the back-end pool of Ubuntu DSVMs. As a user, you would sign in to the VM on a Secure Shell (SSH) or on JupyterHub in the normal way. Because the VM instances can be scaled up or down dynamically, any state must be saved in the mounted Azure Files share. You can use the same approach to create a pool of Windows DSVMs.
48
53
49
-
The [script that mounts the Azure Files share](https://raw.githubusercontent.com/Azure/DataScienceVM/master/Extensions/General/mountazurefiles.sh) is also available in the Azure DataScienceVM repository in GitHub. The script mounts the Azure Files share at the specified mount point in the parameter file. The script also creates soft links to the mounted drive in the initial user's home directory. A user-specific notebook directory in the Azure Files share is soft-linked to the `$HOME/notebooks/remote` directory so that users can access, run, and save their Jupyter notebooks. You can use the same convention when you create additional users on the VM to point each user's Jupyter workspace to the Azure Files share.
54
+
The [script that mounts the Azure Files share](https://raw.githubusercontent.com/Azure/DataScienceVM/master/Extensions/General/mountazurefiles.sh) is also available in the Azure DataScienceVM repository in GitHub. The script mounts the Azure Files share at the specified mount point in the parameter file. The script also creates soft links to the mounted drive in the initial user's home directory. A user-specific notebook directory in the Azure Files share is soft-linked to the `$HOME/notebooks/remote` directory, so that users can access, run, and save their Jupyter notebooks. You can use the same convention when you create more users on the VM, to point each user's Jupyter workspace to the Azure Files share.
50
55
51
-
Virtual machine scale sets support autoscaling. You can set rules about when to create additional instances and when to scale down instances. For example, you can scale down to zero instances to save on cloud hardware usage costs when the VMs are not used at all. The documentation pages of virtual machine scale sets provide detailed steps for [autoscaling](../../virtual-machine-scale-sets/virtual-machine-scale-sets-autoscale-overview.md).
56
+
Virtual machine scale sets support autoscaling. You can set rules about when to create more instances and when to scale down instances. For example, you can scale down to zero instances to save on cloud hardware usage costs when the VMs aren't used at all. The virtual machine scale sets documentation pages provide detailed steps for [autoscaling](../../virtual-machine-scale-sets/virtual-machine-scale-sets-autoscale-overview.md).
0 commit comments