You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hpc-cache/hpc-cache-add-storage.md
+11-8Lines changed: 11 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,13 +4,13 @@ description: How to define storage targets so that your Azure HPC Cache can use
4
4
author: ekpgh
5
5
ms.service: hpc-cache
6
6
ms.topic: conceptual
7
-
ms.date: 12/30/2019
7
+
ms.date: 04/03/2020
8
8
ms.author: rohogue
9
9
---
10
10
11
11
# Add storage targets
12
12
13
-
*Storage targets* are back-end storage for files that are accessed through an Azure HPC Cache instance. You can add NFS storage (like an on-premises hardware system), or store data in Azure Blob.
13
+
*Storage targets* are back-end storage for files that are accessed through an Azure HPC Cache. You can add NFS storage (like an on-premises hardware system), or store data in Azure Blob.
14
14
15
15
You can define up to ten different storage targets for one cache. The cache presents all of the storage targets in one aggregated namespace.
16
16
@@ -30,10 +30,10 @@ A new Blob storage target needs an empty Blob container or a container that is p
30
30
31
31
You can create a new container from this page just before adding it.
32
32
33
-
To define an Azure Blob container, enter this information.
34
-
35
33
36
34
35
+
To define an Azure Blob container, enter this information.
36
+
37
37
***Storage target name** - Set a name that identifies this storage target in the Azure HPC Cache.
38
38
***Target type** - Choose **Blob**.
39
39
***Storage account** - Select the account that you want to use.
@@ -74,7 +74,7 @@ Steps to add the RBAC roles:
74
74
1. In the **Select** field, search for "hpc". This string should match one service principal, named "HPC Cache Resource Provider". Click that principal to select it.
75
75
76
76
> [!NOTE]
77
-
> If a search for "hpc" doesn't work, try using the string "storagecache" instead. Users who joined the previews (before GA) might need to use the older name for the service principal.
77
+
> If a search for "hpc" doesn't work, try using the string "storagecache" instead. Users who participated in previews (before GA) might need to use the older name for the service principal.
78
78
79
79
1. Click the **Save** button at the bottom.
80
80
@@ -86,7 +86,10 @@ Steps to add the RBAC roles:
86
86
87
87
An NFS storage target has more fields than the Blob storage target. These fields specify how to reach the storage export and how to efficiently cache its data. Also, an NFS storage target lets you create multiple namespace paths if the NFS host has more than one export available.
88
88
89
-
89
+
90
+
91
+
> [!NOTE]
92
+
> Before you create an NFS storage target, make sure your storage system is accessible from the Azure HPC Cache and meets permission requirements. Storage target creation will fail if the cache can't access the storage system. Read [NFS storage requirements](hpc-cache-prereqs.md#nfs-storage-requirements) and [Troubleshoot NAS configuration and NFS storage target issues](troubleshoot-nas.md) for details.
90
93
91
94
Provide this information for an NFS-backed storage target:
92
95
@@ -121,7 +124,7 @@ When finished, click **OK** to add the storage target.
121
124
### Choose a usage model
122
125
<!-- referenced from GUI - update aka.ms link if you change this heading -->
123
126
124
-
When you create a storage target that points to an NFS storage system, you need to choose the *usage model* for that target. This model determines how your data is cached.
127
+
When you create a storage target that points to an NFS storage system, you need to choose the usage model for that target. This model determines how your data is cached.
125
128
126
129
There are three options:
127
130
@@ -133,7 +136,7 @@ There are three options:
133
136
134
137
***Greater than 15% writes** - This option speeds up both read and write performance. When using this option, all clients must access files through the Azure HPC Cache instead of mounting the back-end storage directly. The cached files will have recent changes that are not stored on the back end.
135
138
136
-
In this usage model, files in the cache are not checked against the files on back-end storage. The cached version of the file is assumed to be more current. A modified file in the cache is only written to the back-end storage system after it has been in the cache for an hour with no additional changes.
139
+
In this usage model, files in the cache are not checked against the files on back-end storage. The cached version of the file is assumed to be more current. A modified file in the cache is written to the back-end storage system after it has been in the cache for an hour with no additional changes.
137
140
138
141
***Clients write to the NFS target, bypassing the cache** - Choose this option if any clients in your workflow write data directly to the storage system without first writing to the cache. Files that clients request are cached, but any changes to those files from the client are passed back to the back-end storage system immediately.
Copy file name to clipboardExpand all lines: articles/hpc-cache/hpc-cache-mount.md
+82-24Lines changed: 82 additions & 24 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,59 +4,117 @@ description: How to connect clients to an Azure HPC Cache service
4
4
author: ekpgh
5
5
ms.service: hpc-cache
6
6
ms.topic: conceptual
7
-
ms.date: 10/30/2019
7
+
ms.date: 04/03/2020
8
8
ms.author: rohogue
9
9
---
10
10
11
11
# Mount the Azure HPC Cache
12
12
13
-
After the cache is created, NFS clients can access it with a simple mount command.
13
+
After the cache is created, NFS clients can access it with a simple `mount` command. The command connects a specific storage target path on the Azure HPC Cache to a local directory on the client machine.
14
14
15
-
The mount command is made up of two elements:
15
+
The mount command is made up of these elements:
16
16
17
17
* One of the cache's mount addresses (listed on the cache overview page)
18
18
* The virtual namespace path that you set when you created the storage target
19
+
* The local path to use on the client
20
+
* Command parameters that optimize the success of this kind of NFS mount
19
21
20
-
22
+
The **Mount instructions** page for your cache collects the information and the recommended options for you, and creates a prototype mount command that you can copy. Read [Use the mount instructions utility](#use-the-mount-instructions-utility) for details.
21
23
22
-
> [!NOTE]
23
-
> The cache mount addresses correspond to network interfaces inside the cache's subnet. In a resource group, these NICs are listed with names ending in `-cluster-nic-` and a number. Do not alter or delete these interfaces, or the cache will become unavailable.
24
+
## Prepare clients
24
25
25
-
The virtual namespace paths are shown in the **Storage targets** page. Click an individual storage target name to see its details, including aggregated namespace paths associated with it.
26
+
Make sure your clients are able to mount the Azure HPC Cache by following the guidelines in this section.
26
27
27
-
28
+
### Provide network access
29
+
30
+
The client machines must have network access to the cache's virtual network and private subnet.
31
+
32
+
For example, create client VMs within the same virtual network, or use an endpoint, gateway, or other solution in the virtual network for access from outside. (Remember that nothing other than the cache itself should be hosted inside the cache's subnet.)
33
+
34
+
### Install utilities
35
+
36
+
Install the appropriate Linux utility software to support the NFS mount command:
37
+
38
+
* For Red Hat Enterprise Linux or SuSE: `sudo yum install -y nfs-utils`
39
+
* For Ubuntu or Debian: `sudo apt-get install nfs-common`
40
+
41
+
### Create a local path
42
+
43
+
Create a local directory path on each client to connect to the cache. Create a path for each storage target that you want to mount.
44
+
45
+
Example: `sudo mkdir -p /mnt/hpc-cache-1/target3`
46
+
47
+
## Use the mount instructions utility
48
+
49
+
Open the **Mount instructions** page from the **Configure** section of the cache view in the Azure portal.
50
+
51
+
52
+
53
+
The mount command page includes information about the client mount process and prerequisites, plus fields you can use to create a copyable mount command.
54
+
55
+
To use this page, follow this procedure:
56
+
57
+
<!--1. In step one of **Mounting your file system**, enter the path that the client will use to access the Azure HPC Cache storage target.
58
+
59
+
* This path is local to the client.
60
+
* After you provide the directory name, the field populates with a command you can copy. Use this command on the client directly or in a setup script to create the directory path on the client VM. -->
28
61
29
-
## Mount command syntax
62
+
1. Review the client prerequisites and install the utilities needed to use the NFS `mount`command as described above in [Prepare clients](#prepare-clients).
30
63
31
-
Use a mount command like the following:
64
+
1. Step one of **Mounting your file system**<!-- label will change --> gives an example command for creating the local path on the client. This is the path that the client will use to access the content from the Azure HPC Cache.
65
+
66
+
Note the path name so that you can modify it in the command if needed.
67
+
68
+
1. In step two, select one of the available IP addresses. All of the cache's [client mount points](#find-mount-command-components) are listed here. Make sure that you have a system to balance load among all IP addresses.
69
+
70
+
1. The field in step three automatically populates with a prototype mount command. Click the copy symbol at the right side of the field to automatically copy it to your clipboard.
71
+
72
+
> [!NOTE]
73
+
> Check the copy command before using it. You might need to customize the client mount path and the storage target virtual namespace path, which are not yet selectable in this interface. You also should update the mount command options to reflect the [recommended options](#mount-command-options) below. Read [Understand mount command syntax](#understand-mount-command-syntax) for help.
74
+
75
+
1. Use the copied mount command (with edits, if needed) on the client machine to connect it to the storage target on the Azure HPC Cache. You can issue the command directly from the client command line, or include the mount command in a client setup script or template.
76
+
77
+
## Understand mount command syntax
78
+
79
+
The mount command has the following form:
32
80
33
81
> sudo mount *cache_mount_address*:/*namespace_path**local_path* {*options*}
34
82
35
83
Example:
36
84
37
-
```
85
+
```bash
38
86
root@test-client:/tmp# mkdir hpccache
39
-
root@test-client:/tmp# sudo mount 10.0.0.28:/blob-demo-0722 ./hpccache/ -orw,tcp,mountproto=tcp,vers3,hard
40
-
root@test-client:/tmp#
87
+
root@test-client:/tmp# sudo mount 10.0.0.28:/blob-demo-0722 ./hpccache/ -o hard,proto=tcp,mountproto=tcp,retry=30
88
+
root@test-client:/tmp#
41
89
```
42
90
43
-
After this command succeeds, the contents of the storage export should be visible in the ``hpccache`` directory on the client.
44
-
45
-
> [!NOTE]
46
-
> Your clients must be able to access the virtual network and subnet that houses your cache. For example, create client VMs within the same virtual network, or use an endpoint, gateway, or other solution in the virtual network for access from outside. Remember that nothing else can be hosted inside the cache's subnet.
91
+
After this command succeeds, the contents of the storage export will be visible in the ``hpccache`` directory on the client.
47
92
48
93
### Mount command options
49
94
50
-
For a robust client mount, pass these settings and arguments in your mount command:
95
+
For a robust client mount, pass these settings and arguments in your mount command:
> mount -o hard,proto=tcp,mountproto=tcp,retry=30 ${CACHE_IP_ADDRESS}:/${NAMESPACE_PATH} ${LOCAL_FILESYSTEM_MOUNT_POINT}
53
98
54
99
| Recommended mount command settings ||
55
-
--- | ---
56
-
``hard`` | Soft mounts to Azure HPC Cache are associated with application failures and possible data loss.
57
-
``proto=netid`` | This option supports appropriate handling of NFS network errors.
58
-
``mountproto=netid`` | This option supports appropriate handling of network errors for mount operations.
59
-
``retry=n`` | Set ``retry=30`` to avoid transient mount failures. (A different value is recommended in foreground mounts.)
100
+
--- | ---
101
+
``hard`` | Soft mounts to Azure HPC Cache are associated with application failures and possible data loss.
102
+
``proto=tcp`` | This option supports appropriate handling of NFS network errors.
103
+
``mountproto=tcp`` | This option supports appropriate handling of network errors for mount operations.
104
+
``retry=<value>`` | Set ``retry=30`` to avoid transient mount failures. (A different value is recommended in foreground mounts.)
105
+
106
+
### Find mount command components
107
+
108
+
If you want to create a mount command without using the **Mount instructions** page, you can find the mount addresses on the cache **Overview** page and the virtual namespace paths on the **Storage targets** page.
109
+
110
+
111
+
112
+
> [!NOTE]
113
+
> The cache mount addresses correspond to network interfaces inside the cache's subnet. In a resource group, these NICs are listed with names ending in `-cluster-nic-` and a number. Do not alter or delete these interfaces, or the cache will become unavailable.
114
+
115
+
The virtual namespace paths are shown in the **Storage targets** page. Click an individual storage target name to see its details, including aggregated namespace paths associated with it.
116
+
117
+
Copy file name to clipboardExpand all lines: articles/hpc-cache/hpc-cache-prereqs.md
+5-3Lines changed: 5 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ description: Prerequisites for using Azure HPC Cache
4
4
author: ekpgh
5
5
ms.service: hpc-cache
6
6
ms.topic: conceptual
7
-
ms.date: 02/20/2020
7
+
ms.date: 04/03/2020
8
8
ms.author: rohogue
9
9
---
10
10
@@ -108,7 +108,7 @@ More information is included in [Troubleshoot NAS configuration and NFS storage
108
108
109
109
Make sure that all of the ports returned by the ``rpcinfo`` query allow unrestricted traffic from the Azure HPC Cache's subnet.
110
110
111
-
* In addition to the ports returned by the `rpcinfo` command, make sure that these commonly used ports allow inbound and outbound traffic:
111
+
* If you can't use the `rpcinfo` command, make sure that these commonly used ports allow inbound and outbound traffic:
112
112
113
113
| Protocol | Port | Service |
114
114
|----------|-------|----------|
@@ -118,6 +118,8 @@ More information is included in [Troubleshoot NAS configuration and NFS storage
118
118
| TCP/UDP | 4046 | mountd |
119
119
| TCP/UDP | 4047 | status |
120
120
121
+
Some systems use different port numbers for these services - consult your storage system's documentation to be sure.
122
+
121
123
* Check firewall settings to be sure that they allow traffic on all of these required ports. Be sure to check firewalls used in Azure as well as on-premises firewalls in your data center.
122
124
123
125
* **Directory access:** Enable the `showmount` command on the storage system. Azure HPC Cache uses this command to check that your storage target configuration points to a valid export, and also to make sure that multiple mounts don't access the same subdirectories (a risk for file collision).
@@ -127,7 +129,7 @@ More information is included in [Troubleshoot NAS configuration and NFS storage
127
129
128
130
Learn more about directory listing access in the NFS storage target [troubleshooting article](troubleshoot-nas.md#enable-export-listing).
129
131
130
-
* **Root access:** The cache connects to the back-end system as user ID 0. Check these settings on your storage system:
132
+
* **Root access** (read/write): The cache connects to the back-end system as user ID 0. Check these settings on your storage system:
131
133
132
134
* Enable `no_root_squash`. This option ensures that the remote root user can access files owned by root.
0 commit comments