Skip to content

Commit f41e502

Browse files
authored
Merge pull request #87534 from ekpgh/release-preview-hpc-cache
incorporate SME review feedback
2 parents 767fbd1 + f48125c commit f41e502

10 files changed

+143
-161
lines changed

articles/hpc-cache/hpc-cache-add-storage.md

Lines changed: 25 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -4,27 +4,29 @@ description: How to define storage targets so that your Azure HPC Cache can use
44
author: ekpgh
55
ms.service: hpc-cache
66
ms.topic: conceptual
7-
ms.date: 08/30/2019
7+
ms.date: 09/06/2019
88
ms.author: v-erkell
99
---
1010

1111
# Add storage
1212

13-
*Storage targets* are long-term storage for files that are accessed through an Azure HPC Cache instance. You can add NFS storage, like an on-premises hardware system, or store data in Azure Blob.
13+
*Storage targets* are back-end storage for files that are accessed through an Azure HPC Cache instance. You can add NFS storage, like an on-premises hardware system, or store data in Azure Blob.
1414

15-
You can define up to ten different storage targets for one cache. The cache presents all of the storage targets in one client-facing filesystem called the aggregated namespace.
15+
You can define up to ten different storage targets for one cache. The cache presents all of the storage targets in one aggregated namespace.
1616

1717
Remember that the storage exports must be accessible from your cache's virtual network. For on-premises hardware storage, you might need to set up a DNS server that can resolve hostnames for NFS storage access. Read more in [DNS access](hpc-cache-prereqs.md#dns-access).
1818

19-
You can add storage targets while creating your Azure HPC Cache, or afterward. The procedure is slightly different depending on whether you're adding Azure Blob storage or an NFS-accessible export. Details for each are below.
19+
You can add storage targets while creating your Azure HPC Cache, or afterward. The procedure is slightly different depending on whether you're adding Azure Blob storage or an NFS export. Details for each are below.
2020

21-
## Add storage targets while creating the cache
21+
## Add storage targets while creating the cache
2222

2323
Use the **Storage targets** tab of the cache creation wizard to define storage at the same time you create the cache instance.
2424

2525
![screenshot of storage targets page](media/create-targets.png)
2626

27-
## Add storage targets from the cache
27+
Click the **Add storage target** link to add storage.
28+
29+
## Add storage targets from the cache
2830

2931
From the Azure portal, open your cache instance and click **Storage targets** on the left sidebar. The storage target page lists all existing targets and gives a link to add a new one.
3032

@@ -38,26 +40,26 @@ To define an Azure Blob container, enter this information.
3840

3941
* **Storage target name** - Set a name that identifies this storage target in the Azure HPC Cache.
4042
* **Target type** - Choose **Blob**.
41-
* **Storage account** - Select the account with the container to reference.
43+
* **Storage account** - Select the account with the container to reference.
4244

43-
You will need to authorize the cache instance to access the storage account as described in [Add the access role](#add-the-access-control-role-to-your-account).
45+
You will need to authorize the cache instance to access the storage account as described in [Add the access roles](#add-the-access-control-roles-to-your-account).
4446
* **Storage container** - Select the Blob container for this target.
4547

4648
* **Virtual namespace path** - Set the client-facing filepath for this storage target. Read [Configure aggregated namespace](hpc-cache-namespace.md) to learn more about the virtual namespace feature.
4749

48-
The namespace path value must end with a slash (``/``) and should not start with one.
50+
<!-- The namespace path value must end with a slash (``/``) and should not start with one. -->
4951

50-
When finished, click **Ok** to add the storage target.
52+
When finished, click **OK** to add the storage target.
5153

52-
### Add the access control role to your account
54+
### Add the access control roles to your account
5355

5456
The Azure HPC Cache uses [role-based access control (RBAC)](https://docs.microsoft.com/azure/role-based-access-control/index) to authorize the cache application to access your storage account for Azure Blob storage targets.
5557

56-
The storage account owner must explicitly add the role [Storage Account Contributor](https://docs.microsoft.com/azure/role-based-access-control/built-in-roles#storage-account-contributor) for the user "StorageCache Resource Provider".
58+
The storage account owner must explicitly add the rolls [Storage Account Contributor](https://docs.microsoft.com/azure/role-based-access-control/built-in-roles#storage-account-contributor) and [Storage Blob Data Contributor](https://docs.microsoft.com/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor) for the user "StorageCache Resource Provider".
5759

5860
You can do this ahead of time, or by clicking a link on the page where you add a Blob storage target.
5961

60-
Steps to add the RBAC role:
62+
Steps to add the RBAC roles:
6163

6264
1. Open the **Access control (IAM)** page for the storage account. (The link in the **Add storage target** page automatically opens this page for the selected account.)
6365

@@ -67,15 +69,17 @@ Steps to add the RBAC role:
6769

6870
1. In the **Assign access to** field, leave the default value selected ("Azure AD user, group, or service principal").
6971

70-
1. In the **Select** field, search for "storagecache". This string should match one security principal, named "HPC Cache Resource Provider". Click that principal to select it.
72+
1. In the **Select** field, search for "storagecache". This string should match one security principal, named "HPC Cache Resource Provider". Click that principal to select it.
73+
74+
1. Click the **Save** button to add the role assignment to the storage account.
7175

72-
1. Click the **Save** button to add the role assignment to the storage account.
76+
1. Repeat this process to assign the role "Storage Blob Data Contributor".
7377

7478
![screenshot of add role assignment GUI](media/hpc-cache-add-role.png)
7579

7680
## Add a new NFS storage target
7781

78-
An NFS storage target has some extra fields to specify how to reach the storage export and how to efficiently cache its data. Also, you can create multiple storage targets from one NFS host if it has different exports available.
82+
An NFS storage target has some extra fields to specify how to reach the storage export and how to efficiently cache its data. Also, you can create multiple namespace paths from one NFS host if it has more than one export available.
7983

8084
![Screenshot of add storage target page with NFS target defined](media/hpc-cache-add-nfs-target.png)
8185

@@ -91,17 +95,17 @@ Provide this information for an NFS-backed storage target:
9195

9296
You can create multiple namespace paths to represent different exports on the same NFS storage system, but you must create them all from one storage target.
9397

94-
For each export, fill in these values:
98+
For each export, fill in these values:
9599

96100
* **Virtual namespace path** - Set the client-facing filepath for this storage target. Read [Configure aggregated namespace](hpc-cache-namespace.md) to learn more about the virtual namespace feature.
97101

98-
The virtual path should start with a slash ``/``.
102+
<!-- The virtual path should start with a slash ``/``. -->
99103

100-
* **NFS export path** - Enter the path to the data export to mount at this namespace path.
104+
* **NFS export path** - Enter the path to the NFS export.
101105

102-
* **Subdirectory path** - If you want to mount a specific subdirectory of the export, enter it here. If not, enter a slash (``/``).
106+
* **Subdirectory path** - If you want to mount a specific subdirectory of the export, enter it here. If not, leave this field blank.
103107

104-
When finished, click **Ok** to add the storage target.
108+
When finished, click **OK** to add the storage target.
105109

106110
### Choose a usage model
107111
<!-- link in GUI to this heading -->

articles/hpc-cache/hpc-cache-create.md

Lines changed: 33 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -4,130 +4,84 @@ description: How to create an Azure HPC Cache instance
44
author: ekpgh
55
ms.service: hpc-cache
66
ms.topic: tutorial
7-
ms.date: 08/30/2019
7+
ms.date: 09/06/2019
88
ms.author: v-erkell
99
---
1010

11-
# Create a cache
11+
# Create an Azure HPC Cache
1212

13-
Use the Azure portal to create the cache instance.
13+
Use the Azure portal to create your cache.
1414

15-
![screenshot of cache overview in Azure portal, with create button at the bottom](media/hpc-cache-home-page.png)
15+
![screenshot of cache overview in Azure portal, with create button at the bottom](media/hpc-cache-home-page.png)
1616

17-
## Basics
17+
## Define basic details
1818

1919
![screenshot of project details page in Azure portal](media/create-1.png)
2020

21-
In **Project Details**, select the subscription and resource group that will host the Azure HPC Cache. Make sure the subscription is the one that was whitelisted by the preview team.
21+
In **Project Details**, select the subscription and resource group that will host the Azure HPC Cache. Make sure the subscription is on the [preview access](hpc-cache-prereqs.md#azure-subscription) list.
2222

2323
In **Service Details**, set the cache name and these other attributes:
2424

2525
* Location - Select one of the [supported regions](hpc-cache-overview.md#region-availability).
2626
* Virtual network - You can select an existing one or create a new virtual network.
27-
* Subnet - Choose or create a subnet with at least 64 IP addresses (/24) that will be used only for the Azure HPC Cache.
27+
* Subnet - Choose or create a subnet with at least 64 IP addresses (/24) that will be used only for the Azure HPC Cache.
2828

29-
## Set cache size
29+
## Set cache capacity
3030
<!-- change link in GUI -->
3131

32-
On the **Cache** page, you must set the capacity of your Azure HPC Cache.
32+
On the **Cache** page, you must set the capacity of your Azure HPC Cache. This value determines how much data your cache can hold and how quickly it can service client requests. After the public preview period, capacity will also affect the cache's cost.
3333

34-
Cache capacity is determined by two factors:
34+
Cache capacity is measured in input/output operations per second (IOPS). Choose the capacity by setting these two values:
3535

36-
* The number of operations per second that you want the cache to support (throughput)
37-
* The amount of data you want to have room for in the cache storage
36+
* The maximum data transfer rate for the cache (throughput), in GB/second
37+
* The amount of storage allocated for cached data, in TB
3838

39-
Choose one of the available throughput values (in GB/second) and cache storage sizes (in TB). The IOPS capacity is calculated and shown underneath the value selectors.
39+
Choose one of the available throughput values and cache storage sizes. The IOPS capacity is calculated and shown below the value selectors.
40+
41+
Keep in mind that the actual data transfer rate depends on workload, network speeds, and the type of storage targets. If a file isn't in the cache, or it is marked as stale, the service will use some throughput to fetch it from backend storage. The value you choose sets the maximum throughput for the entire cache, and not all of it is available for client requests.
42+
43+
For cache storage, Azure HPC Cache manages which files are cached and preloaded to maximize cache hit rates. The cache contents are continuously assessed and files are moved to long-term storage when they are less frequently accessed. Choose a cache storage size that can comfortably hold the active set of working files with additional space for metadata and other overhead.
4044

4145
![screenshot of cache sizing page](media/create-cache-iops.png)
4246

4347
## Add storage targets
4448

45-
Storage targets are the long-term storage for the contents of your cache. You can define storage targets while creating the cache, but you can also add them afterward with the link in the **Configure** section of your cache's page in the portal.
49+
Storage targets are the back-end, long-term storage for the contents of your cache.
50+
51+
You can define storage targets while creating the cache, but you can also add them afterward with the link in the **Configure** section of your cache's page in the portal.
4652

4753
![screenshot of storage targets page](media/create-targets.png)
4854

49-
Click the **Add storage target link** to define your back-end storage systems. The storage can be Azure Blob containers or on-premises NFS systems.
55+
Click the **Add storage target link** to define your back-end storage systems. The storage can be Azure Blob containers or on-premises NFS systems.
5056

5157
You can define up to ten different storage targets.
5258

5359
For step-by-step instructions to add a storage target, read [Add storage](hpc-cache-add-storage.md). The procedure is different for Blob storage or for NFS exports.
5460

55-
For both types of storage, you must specify the back-end storage system's address or container name and the client-facing namespace path to show for it.
56-
57-
When creating a Blob storage target, make sure the cache has access permissions to the storage account, as described in [Add the access control role](hpc-cache-add-storage.md#add-the-access-control-role-to-your-account).
58-
59-
When creating an NFS storage target, specify a [usage model](hpc-cache-add-storage.md#choose-a-usage-model). The usage model tells the cache how to optimize cache utilization for your workflow.
60-
61-
<!-- ## Storage target details
62-
63-
Slightly different information is used for Blob storage targets than for NFS storage targets.
64-
65-
Both NFS and Blob storage targets require this *common information*:
66-
67-
* **Storage target name** - Set the name that the Azure HPC Cache will display for this storage target.
68-
* **Target type** - Select either Blob or NFS depending on the type of access for the long-term storage.
69-
* **Virtual namespace path** - Set the client-facing filepath for this storage target. Read [Configure aggregated namespace](hpc-cache-namespace.md) to learn more about the virtual namespace feature.
70-
71-
*For an NFS storage target*, you must specify how to reach the storage export and how to efficiently cache its data:
72-
73-
* **Hostname:Export** - Provide the fully qualified domain name (FQDN) and export that the storage target will point to. You can create multiple storage targets to represent different exports on the same NFS storage system.
74-
75-
> [!NOTE]
76-
> Although the example in a screenshot below uses an IP address, FQDN is preferred. An IP address can be used if necessary.
77-
78-
* **Usage model** - Choose one of the data caching profiles based on your workflow, described in [Choose a usage model, below](#choose-a-usage-model-nfs-storage-targets-only).
79-
80-
*For a Blob storage target*, you also need to supply this information:
81-
82-
* **Storage account** - The account that hosts the container
83-
* **Storage container** - The name of the empty container to reference
84-
85-
You can move data to the Blob storage target after the cache is created. Consult your Azure HPC Cache support representative for instructions.
86-
* If you have not already done it, add the access role permission to the storage account as explained [below](#add-the-access-control-role-to-your-account-blob-storage-only).
87-
88-
![screenshot of blob storage target creation form](media/create-add-blob.png)
89-
90-
### Add the access control role to your account (Blob storage only)
91-
92-
The Azure HPC Cache uses [role-based access control (RBAC)](https://docs.microsoft.com/azure/role-based-access-control/index) to authorize the cache application to access your storage account for Azure Blob storage targets.
93-
94-
The storage account owner must explicitly add the role [Storage Account Contributor](https://docs.microsoft.com/azure/role-based-access-control/built-in-roles#storage-account-contributor) for the user "StorageCache Resource Provider".
95-
96-
You can do this when you add the Blob storage target as part of creating the cache instance. Links are provided in the storage target section of the creation wizard.
97-
98-
Steps to add the RBAC role:
61+
For both types of storage, you must specify how to find the back-end storage system (either an NFS address or a Blob container name) and the client-facing namespace path.
9962

100-
1. Open the **Access control (IAM)** page for the storage account.
101-
1. Click the **+** at the top of the page and choose **Add a role assignment**.
102-
1. Select the role "Storage Account Contributor" from the list.
103-
1. In the **Assign access to** field, leave the default value selected ("Azure AD user, group, or service principal").
104-
1. In the **Select** field, search for "storagecache". This string should match one security principal, named HPC Cache Resource Provider. Click that principal to select it.
105-
1. Click the **Save** button to add the role assignment to the storage account.
63+
When creating a Blob storage target, make sure the cache has access permissions to the storage account, as described in [Add the access control roles](hpc-cache-add-storage.md#add-the-access-control-roles-to-your-account). If you are not sure the role configuration will be successful, create the cache first and then add the Blob storage afterward.
10664

107-
![screenshot of add role assignment GUI](media/add-role.png)
65+
When creating an NFS storage target, specify a [usage model](hpc-cache-add-storage.md#choose-a-usage-model). The usage model setting helps the cache optimize your workflow.
10866

109-
### Choose a usage model (NFS storage targets only)
110-
<!-- link in GUI to this heading -->
111-
<!--
67+
## Add resource tags (optional)
11268

113-
When you create a storage target that points to an NFS storage system, you need to choose the *usage model* for that target. This model determines how your data is cached.
69+
The **Tags** page lets you add [resource tags](https://go.microsoft.com/fwlink/?linkid=873112) to your Azure HPC Cache.
11470

115-
* Read heavy - If you mostly use the cache to speed up data read access, choose this option.
116-
* Read/write - If clients use the cache to read and write, choose this option.
117-
* Clients bypass the cache - Choose this option if your clients write data directly to the storage system without first writing to the cache.
71+
## Finish creating the cache
11872

119-
![screenshot of NFS storage target creation form](media/create-add-nfs.png) -->
73+
After configuring the new cache, click the **Review + create** tab. The portal validates your selections and lets you review your choices. If everything is correct, click **Create**.
12074

121-
## Finished cache
75+
Cache creation takes about 10 minutes. You can track the progress in the Azure portal's notifications panel. When it finishes, a notification appears with a link to the new Azure HPC Cache instance.
12276

123-
After about 10 minutes, the Azure HPC Cache is visible in your subscription's **Resources** list.
77+
The cache also is visible in your subscription's **Resources** list.
12478

12579
![screenshot of Azure HPC Cache instance in Azure portal](media/finished-hpc-cache.png)
12680

12781
## Next steps
12882

129-
After your cache appears in the **Resources** list, you can mount it for access, use it to move your working set data to a new Azure Blob storage target, or define additional data sources.
83+
After your cache appears in the **Resources** list, you can mount it for client access, use it to move your working set data to a new Azure Blob storage target, or define additional data sources.
13084

131-
* [Access the Azure HPC Cache system](hpc-cache-mount.md)
85+
* [Mount the Azure HPC Cache](hpc-cache-mount.md)
13286
* [Move data to Azure Blob storage for Azure HPC Cache](hpc-cache-ingest.md)
133-
* [Add storage targets](hpc-cache-add-storage.md)
87+
* [Add storage targets](hpc-cache-add-storage.md)

articles/hpc-cache/hpc-cache-ingest-parallelcp.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ sudo apt install -y gcc bison gcc binutils make parallel
8282
cd
8383
wget https://mirrors.kernel.org/gnu/libc/glibc-2.27.tar.bz2
8484
tar jxf glibc-2.27.tar.bz2
85-
ln -s /nfs/cache1 hpccache
85+
ln -s /nfs/cache1 hpccache
8686
time parallelcp glibc-2.27 avere/glibc-2.27
8787
cd
8888
mkdir obj

0 commit comments

Comments
 (0)