Skip to content

Commit be59197

Browse files
huypubhuypub
authored
Merge pull request #87542 from MicrosoftDocs/release-preview-hpc-cache
Release preview hpc cache
2 parents 7f1c49f + 96e3c4c commit be59197

27 files changed

+1057
-0
lines changed

‎articles/hpc-cache/hpc-cache-add-storage.md

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
---
2+
title: Add storage to an Azure HPC Cache
3+
description: How to define storage targets so that your Azure HPC Cache can use your on-premises NFS system or Azure Blob containers for long-term file storage
4+
author: ekpgh
5+
ms.service: hpc-cache
6+
ms.topic: conceptual
7+
ms.date: 09/06/2019
8+
ms.author: v-erkell
9+
---
10+
11+
# Add storage
12+
13+
*Storage targets* are back-end storage for files that are accessed through an Azure HPC Cache instance. You can add NFS storage, like an on-premises hardware system, or store data in Azure Blob.
14+
15+
You can define up to ten different storage targets for one cache. The cache presents all of the storage targets in one aggregated namespace.
16+
17+
Remember that the storage exports must be accessible from your cache's virtual network. For on-premises hardware storage, you might need to set up a DNS server that can resolve hostnames for NFS storage access. Read more in [DNS access](hpc-cache-prereqs.md#dns-access).
18+
19+
You can add storage targets while creating your Azure HPC Cache, or afterward. The procedure is slightly different depending on whether you're adding Azure Blob storage or an NFS export. Details for each are below.
20+
21+
## Add storage targets while creating the cache
22+
23+
Use the **Storage targets** tab of the cache creation wizard to define storage at the same time you create the cache instance.
24+
25+
![screenshot of storage targets page](media/create-targets.png)
26+
27+
Click the **Add storage target** link to add storage.
28+
29+
## Add storage targets from the cache
30+
31+
From the Azure portal, open your cache instance and click **Storage targets** on the left sidebar. The storage target page lists all existing targets and gives a link to add a new one.
32+
33+
## Add a new Azure Blob storage target
34+
35+
A new Blob storage target needs an empty Blob container or a container that is populated with data in the Azure HPC Cache cloud filesystem format. Read more about pre-loading a Blob container in [Move data to Azure Blob storage](hpc-cache-ingest.md).
36+
37+
To define an Azure Blob container, enter this information.
38+
39+
![screenshot of the add storage target page, populated with information for a new Azure Blob storage target](media/hpc-cache-add-blob.png)
40+
41+
* **Storage target name** - Set a name that identifies this storage target in the Azure HPC Cache.
42+
* **Target type** - Choose **Blob**.
43+
* **Storage account** - Select the account with the container to reference.
44+
45+
You will need to authorize the cache instance to access the storage account as described in [Add the access roles](#add-the-access-control-roles-to-your-account).
46+
* **Storage container** - Select the Blob container for this target.
47+
48+
* **Virtual namespace path** - Set the client-facing filepath for this storage target. Read [Configure aggregated namespace](hpc-cache-namespace.md) to learn more about the virtual namespace feature.
49+
50+
<!-- The namespace path value must end with a slash (``/``) and should not start with one. -->
51+
52+
When finished, click **OK** to add the storage target.
53+
54+
### Add the access control roles to your account
55+
56+
The Azure HPC Cache uses [role-based access control (RBAC)](https://docs.microsoft.com/azure/role-based-access-control/index) to authorize the cache application to access your storage account for Azure Blob storage targets.
57+
58+
The storage account owner must explicitly add the rolls [Storage Account Contributor](https://docs.microsoft.com/azure/role-based-access-control/built-in-roles#storage-account-contributor) and [Storage Blob Data Contributor](https://docs.microsoft.com/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor) for the user "StorageCache Resource Provider".
59+
60+
You can do this ahead of time, or by clicking a link on the page where you add a Blob storage target.
61+
62+
Steps to add the RBAC roles:
63+
64+
1. Open the **Access control (IAM)** page for the storage account. (The link in the **Add storage target** page automatically opens this page for the selected account.)
65+
66+
1. Click the **+** at the top of the page and choose **Add a role assignment**.
67+
68+
1. Select the role "Storage Account Contributor" from the list.
69+
70+
1. In the **Assign access to** field, leave the default value selected ("Azure AD user, group, or service principal").
71+
72+
1. In the **Select** field, search for "storagecache". This string should match one security principal, named "HPC Cache Resource Provider". Click that principal to select it.
73+
74+
1. Click the **Save** button to add the role assignment to the storage account.
75+
76+
1. Repeat this process to assign the role "Storage Blob Data Contributor".
77+
78+
![screenshot of add role assignment GUI](media/hpc-cache-add-role.png)
79+
80+
## Add a new NFS storage target
81+
82+
An NFS storage target has some extra fields to specify how to reach the storage export and how to efficiently cache its data. Also, you can create multiple namespace paths from one NFS host if it has more than one export available.
83+
84+
![Screenshot of add storage target page with NFS target defined](media/hpc-cache-add-nfs-target.png)
85+
86+
Provide this information for an NFS-backed storage target:
87+
88+
* **Storage target name** - Set a name that identifies this storage target in the Azure HPC Cache.
89+
90+
* **Target type** - Choose **NFS**.
91+
92+
* **Hostname** - Enter the IP address or fully qualified domain name for your NFS storage system. (Use a domain name only if your cache has access to a DNS server that can resolve the name.)
93+
94+
* **Usage model** - Choose one of the data caching profiles based on your workflow, described in [Choose a usage model, below](#choose-a-usage-model).
95+
96+
You can create multiple namespace paths to represent different exports on the same NFS storage system, but you must create them all from one storage target.
97+
98+
For each export, fill in these values:
99+
100+
* **Virtual namespace path** - Set the client-facing filepath for this storage target. Read [Configure aggregated namespace](hpc-cache-namespace.md) to learn more about the virtual namespace feature.
101+
102+
<!-- The virtual path should start with a slash ``/``. -->
103+
104+
* **NFS export path** - Enter the path to the NFS export.
105+
106+
* **Subdirectory path** - If you want to mount a specific subdirectory of the export, enter it here. If not, leave this field blank.
107+
108+
When finished, click **OK** to add the storage target.
109+
110+
### Choose a usage model
111+
<!-- link in GUI to this heading -->
112+
113+
When you create a storage target that points to an NFS storage system, you need to choose the *usage model* for that target. This model determines how your data is cached.
114+
115+
* Read heavy - If you mostly use the cache to speed up data read access, choose this option.
116+
117+
* Read/write - If clients use the cache to read and write, choose this option.
118+
119+
* Clients bypass the cache - Choose this option if your clients write data directly to the storage system without first writing to the cache.
120+
121+
## Next steps
122+
123+
After creating storage targets, consider one of these tasks:
124+
125+
* [Mount the Azure HPC Cache](hpc-cache-mount.md)
126+
* [Move data to Azure Blob storage](hpc-cache-ingest.md)

‎articles/hpc-cache/hpc-cache-create.md

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
---
2+
title: Create an Azure HPC Cache
3+
description: How to create an Azure HPC Cache instance
4+
author: ekpgh
5+
ms.service: hpc-cache
6+
ms.topic: tutorial
7+
ms.date: 09/06/2019
8+
ms.author: v-erkell
9+
---
10+
11+
# Create an Azure HPC Cache
12+
13+
Use the Azure portal to create your cache.
14+
15+
![screenshot of cache overview in Azure portal, with create button at the bottom](media/hpc-cache-home-page.png)
16+
17+
## Define basic details
18+
19+
![screenshot of project details page in Azure portal](media/create-1.png)
20+
21+
In **Project Details**, select the subscription and resource group that will host the Azure HPC Cache. Make sure the subscription is on the [preview access](hpc-cache-prereqs.md#azure-subscription) list.
22+
23+
In **Service Details**, set the cache name and these other attributes:
24+
25+
* Location - Select one of the [supported regions](hpc-cache-overview.md#region-availability).
26+
* Virtual network - You can select an existing one or create a new virtual network.
27+
* Subnet - Choose or create a subnet with at least 64 IP addresses (/24) that will be used only for the Azure HPC Cache.
28+
29+
## Set cache capacity
30+
<!-- change link in GUI -->
31+
32+
On the **Cache** page, you must set the capacity of your Azure HPC Cache. This value determines how much data your cache can hold and how quickly it can service client requests. After the public preview period, capacity will also affect the cache's cost.
33+
34+
Cache capacity is measured in input/output operations per second (IOPS). Choose the capacity by setting these two values:
35+
36+
* The maximum data transfer rate for the cache (throughput), in GB/second
37+
* The amount of storage allocated for cached data, in TB
38+
39+
Choose one of the available throughput values and cache storage sizes. The IOPS capacity is calculated and shown below the value selectors.
40+
41+
Keep in mind that the actual data transfer rate depends on workload, network speeds, and the type of storage targets. If a file isn't in the cache, or it is marked as stale, the service will use some throughput to fetch it from backend storage. The value you choose sets the maximum throughput for the entire cache, and not all of it is available for client requests.
42+
43+
For cache storage, Azure HPC Cache manages which files are cached and preloaded to maximize cache hit rates. The cache contents are continuously assessed and files are moved to long-term storage when they are less frequently accessed. Choose a cache storage size that can comfortably hold the active set of working files with additional space for metadata and other overhead.
44+
45+
![screenshot of cache sizing page](media/create-cache-iops.png)
46+
47+
## Add storage targets
48+
49+
Storage targets are the back-end, long-term storage for the contents of your cache.
50+
51+
You can define storage targets while creating the cache, but you can also add them afterward with the link in the **Configure** section of your cache's page in the portal.
52+
53+
![screenshot of storage targets page](media/create-targets.png)
54+
55+
Click the **Add storage target link** to define your back-end storage systems. The storage can be Azure Blob containers or on-premises NFS systems.
56+
57+
You can define up to ten different storage targets.
58+
59+
For step-by-step instructions to add a storage target, read [Add storage](hpc-cache-add-storage.md). The procedure is different for Blob storage or for NFS exports.
60+
61+
For both types of storage, you must specify how to find the back-end storage system (either an NFS address or a Blob container name) and the client-facing namespace path.
62+
63+
When creating a Blob storage target, make sure the cache has access permissions to the storage account, as described in [Add the access control roles](hpc-cache-add-storage.md#add-the-access-control-roles-to-your-account). If you are not sure the role configuration will be successful, create the cache first and then add the Blob storage afterward.
64+
65+
When creating an NFS storage target, specify a [usage model](hpc-cache-add-storage.md#choose-a-usage-model). The usage model setting helps the cache optimize your workflow.
66+
67+
## Add resource tags (optional)
68+
69+
The **Tags** page lets you add [resource tags](https://go.microsoft.com/fwlink/?linkid=873112) to your Azure HPC Cache.
70+
71+
## Finish creating the cache
72+
73+
After configuring the new cache, click the **Review + create** tab. The portal validates your selections and lets you review your choices. If everything is correct, click **Create**.
74+
75+
Cache creation takes about 10 minutes. You can track the progress in the Azure portal's notifications panel. When it finishes, a notification appears with a link to the new Azure HPC Cache instance.
76+
77+
The cache also is visible in your subscription's **Resources** list.
78+
79+
![screenshot of Azure HPC Cache instance in Azure portal](media/finished-hpc-cache.png)
80+
81+
## Next steps
82+
83+
After your cache appears in the **Resources** list, you can mount it for client access, use it to move your working set data to a new Azure Blob storage target, or define additional data sources.
84+
85+
* [Mount the Azure HPC Cache](hpc-cache-mount.md)
86+
* [Move data to Azure Blob storage for Azure HPC Cache](hpc-cache-ingest.md)
87+
* [Add storage targets](hpc-cache-add-storage.md)

0 commit comments

Comments
 (0)