You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/avere-vfxt/avere-vfxt-add-storage.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ description: How to add a back-end storage system to your Avere vFXT for Azure
4
4
author: ekpgh
5
5
ms.service: avere-vfxt
6
6
ms.topic: conceptual
7
-
ms.date: 12/14/2019
7
+
ms.date: 01/13/2020
8
8
ms.author: rohogue
9
9
---
10
10
@@ -27,7 +27,7 @@ These steps use the Avere Control Panel. Read [Access the vFXT cluster](avere-vf
27
27
28
28
## Create a core filer
29
29
30
-
"Core filer" is a vFXT term for a backend storage system. The storage can be a hardware NAS appliance like NetApp or Isilon, or it can be a cloud object store. More information about core filers can be found in the [Avere cluster settings guide](https://azure.github.io/Avere/legacy/ops_guide/4_7/html/settings_overview.html#managing-core-filers).
30
+
"Core filer" is a vFXT term for a back-end storage system. The storage can be a hardware NAS appliance like NetApp or Isilon, or it can be a cloud object store. More information about core filers can be found in the [Avere cluster settings guide](https://azure.github.io/Avere/legacy/ops_guide/4_7/html/settings_overview.html#managing-core-filers).
31
31
32
32
To add a core filer, choose one of the two main types of core filers:
33
33
@@ -64,7 +64,7 @@ Next, proceed to [Create a junction](#create-a-junction).
64
64
65
65
### Azure Blob Storage cloud core filer
66
66
67
-
To use Azure Blob storage as your vFXT cluster's backend storage, you need an empty container to add as a core filer.
67
+
To use Azure Blob storage as your vFXT cluster's back-end storage, you need an empty container to add as a core filer.
68
68
69
69
Adding Blob storage to your cluster requires these tasks:
Copy file name to clipboardExpand all lines: articles/avere-vfxt/avere-vfxt-additional-resources.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ description: Links to additional information about Avere vFXT for Azure
4
4
author: ekpgh
5
5
ms.service: avere-vfxt
6
6
ms.topic: conceptual
7
-
ms.date: 01/09/2019
7
+
ms.date: 01/13/2020
8
8
ms.author: rohogue
9
9
---
10
10
@@ -20,12 +20,12 @@ Additional Avere cluster documentation can be found on the website at <https://a
20
20
*[Customizing Support and Monitoring Settings](<https://azure.github.io/Avere/legacy/create_cluster/4_8/html/config_support.html#config-support>) explains how to customize support upload settings and enable remote monitoring.
21
21
*[Configuring VServers and Global Namespace](<https://azure.github.io/Avere/legacy/create_cluster/4_8/html/config_vserver.html#config-vserver>) has information about creating a client-facing namespace.
22
22
*[Configuring DNS for the Avere cluster](<https://azure.github.io/Avere/legacy/create_cluster/4_8/html/config_network.html#dns-overview>) explains how to configure round-robin DNS.
23
-
*[Adding Backend Storage](<https://azure.github.io/Avere/legacy/create_cluster/4_8/html/config_core_filer.html#add-core-filer>) documents how to add core filers.
23
+
*[Adding Back-end Storage](<https://azure.github.io/Avere/legacy/create_cluster/4_8/html/config_core_filer.html#add-core-filer>) documents how to add core filers.
24
24
25
25
* The [Cluster Configuration Guide](<https://azure.github.io/Avere/#operations>) is a complete reference of settings and options for an Avere cluster. A vFXT cluster uses a subset of these options, but most of the same configuration pages apply.
26
26
27
27
* The [Dashboard Guide](<https://azure.github.io/Avere/#operations>) explains how to use the cluster monitoring features of the Avere Control Panel.
28
28
29
29
## vFXT creation and management documentation
30
30
31
-
A full guide for using vfxt.py, the cloud cluster creation and management utility, is provided on GitHub: [Cloud cluster management with vfxt.py](https://github.com/Azure/AvereSDK/blob/master/docs/README.md).
31
+
A full guide for using vfxt.py, a script-based cloud cluster creation and management utility, is provided on GitHub: [Cloud cluster management with vfxt.py](https://github.com/Azure/AvereSDK/blob/master/docs/README.md).
Copy file name to clipboardExpand all lines: articles/avere-vfxt/avere-vfxt-data-ingest.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,15 +4,15 @@ description: How to add data to a new storage volume for use with the Avere vFXT
4
4
author: ekpgh
5
5
ms.service: avere-vfxt
6
6
ms.topic: conceptual
7
-
ms.date: 11/21/2019
7
+
ms.date: 12/16/2019
8
8
ms.author: rohogue
9
9
---
10
10
11
11
# Moving data to the vFXT cluster - Parallel data ingest
12
12
13
-
After you've created a new vFXT cluster, your first task might be to move data onto its new storage volume. However, if your usual method of moving data is issuing a simple copy command from one client, you will likely see a slow copy performance. Single-threaded copying is not a good option for copying data to the Avere vFXT cluster's backend storage.
13
+
After you've created a new vFXT cluster, your first task might be to move data onto a new storage volume in Azure. However, if your usual method of moving data is issuing a simple copy command from one client, you will likely see a slow copy performance. Single-threaded copying is not a good option for copying data to the Avere vFXT cluster's back-end storage.
14
14
15
-
Because the Avere vFXT cluster is a scalable multi-client cache, the fastest and most efficient way to copy data to it is with multiple clients. This technique parallelizes ingestion of the files and objects.
15
+
Because the Avere vFXT for Azure cluster is a scalable multi-client cache, the fastest and most efficient way to copy data to it is with multiple clients. This technique parallelizes ingestion of the files and objects.
16
16
17
17

18
18
@@ -39,12 +39,12 @@ The data ingestor VM is part of a tutorial where the newly created VM mounts the
39
39
40
40
## Strategic planning
41
41
42
-
When building a strategy to copy data in parallel, you should understand the tradeoffs in file size, file count, and directory depth.
42
+
When designing a strategy to copy data in parallel, you should understand the tradeoffs in file size, file count, and directory depth.
43
43
44
44
* When files are small, the metric of interest is files per second.
45
45
* When files are large (10MiBi or greater), the metric of interest is bytes per second.
46
46
47
-
Each copy process has a throughput rate and a files-transferred rate, which can be measured by timing the length of the copy command and factoring the file size and file count. Explaining how to measure the rates is outside the scope of this document, but it is imperative to understand whether you’ll be dealing with small or large files.
47
+
Each copy process has a throughput rate and a files-transferred rate, which can be measured by timing the length of the copy command and factoring the file size and file count. Explaining how to measure the rates is outside the scope of this document, but it is important to understand whether you’ll be dealing with small or large files.
48
48
49
49
## Manual copy example
50
50
@@ -273,7 +273,7 @@ This method is a simple and time-effective method for datasets up to the number
273
273
274
274
## Use the msrsync utility
275
275
276
-
The ``msrsync`` tool also can be used to move data to a backend core filer for the Avere cluster. This tool is designed to optimize bandwidth usage by running multiple parallel ``rsync`` processes. It is available from GitHub at <https://github.com/jbd/msrsync>.
276
+
The ``msrsync`` tool also can be used to move data to a back-end core filer for the Avere cluster. This tool is designed to optimize bandwidth usage by running multiple parallel ``rsync`` processes. It is available from GitHub at <https://github.com/jbd/msrsync>.
277
277
278
278
``msrsync`` breaks up the source directory into separate “buckets” and then runs individual ``rsync`` processes on each bucket.
279
279
@@ -318,7 +318,7 @@ To use ``msrsync`` to populate an Azure cloud volume with an Avere cluster, foll
318
318
319
319
## Use the parallel copy script
320
320
321
-
The ``parallelcp`` script also can be useful for moving data to your vFXT cluster's backend storage.
321
+
The ``parallelcp`` script also can be useful for moving data to your vFXT cluster's back-end storage.
322
322
323
323
The script below will add the executable `parallelcp`. (This script is designed for Ubuntu; if using another distribution, you must install ``parallel`` separately.)
Copy file name to clipboardExpand all lines: articles/avere-vfxt/avere-vfxt-demo-links.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ description: links to code demonstration tutorials for Avere vFXT
4
4
author: ekpgh
5
5
ms.service: avere-vfxt
6
6
ms.topic: conceptual
7
-
ms.date: 10/31/2018
7
+
ms.date: 12/19/2019
8
8
ms.author: rohogue
9
9
---
10
10
@@ -14,13 +14,13 @@ Sample tutorials are provided on [GitHub](https://github.com/Azure/Avere). These
14
14
15
15
## Video rendering
16
16
17
-
*[Rendering using Azure Batch and Avere vFXT](https://github.com/Azure/Avere/blob/master/docs/maya_azure_batch_avere_vfxt_demo.md) - A 60-minute demo that demonstrates how to use Autodesk Maya with Azure Batch and an Avere vFXT cluster to generate an animated movie
17
+
*[Rendering using Azure Batch and Avere vFXT](https://github.com/Azure/Avere/blob/master/docs/maya_azure_batch_avere_vfxt_demo.md) - A 60-minute project that demonstrates how to use Autodesk Maya with Azure Batch and an Avere vFXT cluster to generate an animated movie
18
18
19
-
*[Why use the Avere vFXT for rendering?](https://github.com/Azure/Avere/blob/master/docs/why_avere_for_rendering.md) - A demonstration that compares rendering times from NAS with and without an Avere vFXT cluster
19
+
*[Why use the Avere vFXT for rendering?](https://github.com/Azure/Avere/blob/master/docs/why_avere_for_rendering.md) - A demonstration that compares rendering times from network-attached storage with and without an Avere vFXT cluster
20
20
21
21
## High-performance computing
22
22
23
-
*[Best Practices for Improving Azure Virtual Machine (VM) Boot Time](https://github.com/Azure/Avere/blob/master/docs/azure_vm_provision_best_practices.md) - A reproducible test of using Avere vFXT for Azure to improve boot times when spinning up thousands of compute clients quickly
23
+
*[Best practices for improving Azure Virtual Machine (VM) boot time](https://github.com/Azure/Avere/blob/master/docs/azure_vm_provision_best_practices.md) - A reproducible test that uses Avere vFXT for Azure to improve boot times when spinning up thousands of compute clients quickly
Copy file name to clipboardExpand all lines: articles/avere-vfxt/avere-vfxt-deploy-overview.md
+10-7Lines changed: 10 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,9 +4,10 @@ description: Overview of deploying Avere vFXT for Azure
4
4
author: ekpgh
5
5
ms.service: avere-vfxt
6
6
ms.topic: conceptual
7
-
ms.date: 02/20/2019
7
+
ms.date: 01/13/2020
8
8
ms.author: rohogue
9
9
---
10
+
<!-- filename is linked to in the marketplace template, make sure it gets a redirect if we rename it -->
10
11
11
12
# Avere vFXT for Azure - deployment overview
12
13
@@ -20,7 +21,7 @@ After [planning your system](avere-vfxt-deploy-plan.md), you can begin to create
20
21
21
22
An Azure Resource Manager template in the Azure Marketplace collects the necessary information and automatically deploys the entire cluster.
22
23
23
-
After the vFXT cluster is up and running, you will want to know how to connect clients to it and (optionally) how to move your data to the new Blob storage container. If you use a NAS storage system, you need to add it after the cluster is created.
24
+
After the vFXT cluster is up and running, there are still some configuration steps to take before using it. If you created a new Blob storage container, you'll want to move your data to it. If you use a NAS storage system, you need to add it after the cluster is created. You will want to connect clients to the cluster.
24
25
25
26
Here is an overview of all of the steps.
26
27
@@ -35,21 +36,23 @@ Here is an overview of all of the steps.
35
36
Cluster creation involves these steps, which are all done by the marketplace template:
36
37
37
38
* Create new network infrastructure and resource groups, if needed
38
-
* Create a *cluster controller*
39
+
* Create a cluster controller
39
40
40
41
The cluster controller is a simple VM that resides in the same virtual network as the Avere vFXT cluster and has the custom software needed to create and manage the cluster. The controller creates the vFXT nodes and forms the cluster, and it also provides a command-line interface to manage the cluster during its lifetime.
41
42
42
-
If you create a new virtual network or subnet during the deployment, your controller will have a public IP address. This means the controller can serve as a jump host for connecting to the cluster from outside the subnet.
43
+
If you create a new virtual network or subnet during the deployment, your controller will have a public IP address. This means the controller can serve as a jump host for connecting to the cluster from outside the virtual network.
43
44
44
45
* Create the cluster node VMs
45
46
46
-
*Configure the cluster node VMs to form the cluster
47
+
*Create the cluster from the individual nodes
47
48
48
49
* Optionally, create a new Blob container and configure it as back-end storage for the cluster
49
50
51
+
Cluster creation is described in detail in [Deploy the vFXT cluster](avere-vfxt-deploy.md).
52
+
50
53
1. Configure the cluster
51
54
52
-
Connect to the Avere vFXT configuration interface (Avere Control Panel) to customize the cluster's settings. Opt in for support monitoring, and add your storage system if you are using an on-premises data center.
55
+
Connect to the Avere vFXT configuration interface (Avere Control Panel) to customize the cluster's settings. Opt in for support monitoring, and add your storage system if you are using hardware storage or additional Blob containers.
53
56
54
57
*[Access the vFXT cluster](avere-vfxt-cluster-gui.md)
55
58
*[Enable support](avere-vfxt-enable-support.md)
@@ -62,7 +65,7 @@ Here is an overview of all of the steps.
62
65
1. Add data (if needed)
63
66
64
67
Because the Avere vFXT is a scalable multi-client cache, the best way to move data to a new back-end storage container is with a multi-client, multi-threaded file copy strategy.
65
-
68
+
66
69
If you need to move working set data to a new Blob container or other back-end storage system, follow the instructions in [Moving data to the vFXT cluster](avere-vfxt-data-ingest.md).
0 commit comments