Skip to content

Commit cc0e5e4

Browse files
v-shilsv-shils
authored
Merge pull request #77870 from alkohli/5heavy
Tutorial to order Data Box Heavy
2 parents 900c1f3 + 0c755ce commit cc0e5e4

40 files changed

+724
-1
lines changed

‎articles/databox/TOC.yml

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@
112112
items:
113113
- name: Manage via Azure portal
114114
href: data-box-portal-ui-admin.md
115-
- name: Troubleshoot disk issues
115+
- name: Troubleshoot Data Box Heavy issues
116116
href: data-box-disk-troubleshoot.md
117117
- name: Contact Support
118118
href: data-box-disk-contact-microsoft-support.md
@@ -123,12 +123,32 @@
123123
items:
124124
- name: What is Data Box Heavy?
125125
href: data-box-heavy-overview.md
126+
- name: Tutorials
127+
items:
128+
- name: 1 - Order
129+
href: data-box-heavy-deploy-ordered.md
130+
- name: 2 - Set up
131+
href: data-box-heavy-deploy-set-up.md
132+
- name: 3 - Connect & copy
133+
items:
134+
- name: Via SMB
135+
href: data-box-heavy-deploy-copy-data.md
136+
- name: 4 - Return, upload, verify
137+
href: data-box-heavy-deploy-picked-up.md
126138
- name: Concepts
127139
items:
128140
- name: Review requirements
129141
href: data-box-heavy-system-requirements.md
130142
- name: Understand limits
131143
href: data-box-heavy-limits.md
144+
- name: How to
145+
items:
146+
- name: Manage via Azure portal
147+
href: data-box-portal-admin.md
148+
- name: Troubleshoot device issues
149+
href: data-box-troubleshoot.md
150+
- name: Contact Support
151+
href: data-box-disk-contact-microsoft-support.md
132152
- name: Resources
133153
items:
134154
- name: Data Box product

‎articles/databox/data-box-heavy-deploy-copy-data.md

Lines changed: 255 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,255 @@
1+
---
2+
title: Tutorial to copy data via SMB on Azure Data Box Heavy | Microsoft Docs
3+
description: Learn how to copy data to your Azure Data Box Heavy via SMB
4+
services: databox
5+
author: alkohli
6+
7+
ms.service: databox
8+
ms.subservice: heavy
9+
ms.topic: tutorial
10+
ms.date: 05/28/2019
11+
ms.author: alkohli
12+
#Customer intent: As an IT admin, I need to be able to copy data to Data Box Heavy to upload on-premises data from my server onto Azure.
13+
---
14+
# Tutorial: Copy data to Azure Data Box Heavy via SMB
15+
16+
This tutorial describes how to connect to and copy data from your host computer using the local web UI.
17+
18+
In this tutorial, you learn how to:
19+
20+
> [!div class="checklist"]
21+
> * Connect to Data Box Heavy
22+
> * Copy data to Data Box Heavy
23+
24+
25+
## Prerequisites
26+
27+
Before you begin, make sure that:
28+
29+
1. You've completed the [Tutorial: Set up Azure Data Box Heavy](data-box-deploy-set-up.md).
30+
2. You've received your Data Box Heavy and the order status in the portal is **Delivered**.
31+
3. You have a host computer that has the data that you want to copy over to Data Box Heavy. Your host computer must
32+
- Run a [Supported operating system](data-box-system-requirements.md).
33+
- Be connected to a high-speed network. For fastest copy speeds, two 40-GbE connections (one per node) can be utilized in parallel. If you do not have 40-GbE connection available, we recommend that you have at least two 10-GbE connections (one per node).
34+
35+
## Connect to Data Box Heavy shares
36+
37+
Based on the storage account selected, Data Box Heavy creates up to:
38+
- Three shares for each associated storage account for GPv1 and GPv2.
39+
- One share for premium storage.
40+
- One share for blob storage account.
41+
42+
These shares are created on both the nodes of the device.
43+
44+
Under block blob and page blob shares:
45+
- First-level entities are containers.
46+
- Second-level entities are blobs.
47+
48+
Under shares for Azure Files:
49+
- First-level entities are shares.
50+
- Second-level entities are files.
51+
52+
The following table shows the UNC path to the shares on your Data Box Heavy and Azure Storage path URL where the data is uploaded. The final Azure Storage path URL can be derived from the UNC share path.
53+
54+
| | |
55+
|-------------------|--------------------------------------------------------------------------------|
56+
| Azure Block blobs | <li>UNC path to shares: `\\<DeviceIPAddress>\<StorageAccountName_BlockBlob>\<ContainerName>\files\a.txt`</li><li>Azure Storage URL: `https://<StorageAccountName>.blob.core.windows.net/<ContainerName>/files/a.txt`</li> |
57+
| Azure Page blobs | <li>UNC path to shares: `\\<DeviceIPAddres>\<StorageAccountName_PageBlob>\<ContainerName>\files\a.txt`</li><li>Azure Storage URL: `https://<StorageAccountName>.blob.core.windows.net/<ContainerName>/files/a.txt`</li> |
58+
| Azure Files |<li>UNC path to shares: `\\<DeviceIPAddres>\<StorageAccountName_AzFile>\<ShareName>\files\a.txt`</li><li>Azure Storage URL: `https://<StorageAccountName>.file.core.windows.net/<ShareName>/files/a.txt`</li> |
59+
60+
The steps to connect using a Windows or a Linux client are different.
61+
62+
> [!NOTE]
63+
> Follow the same steps to connect to both the nodes of the device in parallel.
64+
65+
### Connect on a Windows system
66+
67+
If using a Windows Server host computer, follow these steps to connect to the Data Box Heavy.
68+
69+
1. The first step is to authenticate and start a session. Go to **Connect and copy**. Click **Get credentials** to get the access credentials for the shares associated with your storage account.
70+
71+
![Get share credentials 1](media/data-box-heavy-deploy-copy-data/get-share-credentials-1.png)
72+
73+
2. In the Access share and copy data dialog box, copy the **Username** and the **Password** corresponding to the share. Click **OK**.
74+
75+
![Get share credentials 1](media/data-box-heavy-deploy-copy-data/get-share-credentials-2.png)
76+
77+
3. To access the shares associated with your storage account (*databoxe2etest* in the following example) from your host computer, open a command window. At the command prompt, type:
78+
79+
`net use \\<IP address of the device>\<share name> /u:<user name for the share>`
80+
81+
Depending upon your data format, the share paths are as follows:
82+
- Azure Block blob - `\\10.100.10.100\databoxe2etest_BlockBlob`
83+
- Azure Page blob - `\\10.100.10.100\databoxe2etest_PageBlob`
84+
- Azure Files - `\\10.100.10.100\databoxe2etest_AzFile`
85+
86+
4. Enter the password for the share when prompted. The following sample shows connecting to a share via the preceding command.
87+
88+
```
89+
C:\Users\Databoxuser>net use \\10.100.10.100\databoxe2etest_BlockBlob /u:databoxe2etest
90+
Enter the password for 'databoxe2etest' to connect to '10.100.10.100':
91+
The command completed successfully.
92+
```
93+
94+
4. Press Windows + R. In the **Run** window, specify the `\\<device IP address>`. Click **OK** to open File Explorer.
95+
96+
![Connect to share via File Explorer 2](media/data-box-heavy-deploy-copy-data/connect-shares-file-explorer-1.png)
97+
98+
You should now see the shares as folders.
99+
100+
![Connect to share via File Explorer 2](media/data-box-heavy-deploy-copy-data/connect-shares-file-explorer-2.png)
101+
102+
**Always create a folder for the files that you intend to copy under the share and then copy the files to that folder**. The folder created under block blob and page blob shares represents a container to which data is uploaded as blobs. You cannot copy files directly to *root* folder in the storage account.
103+
104+
### Connect on a Linux system
105+
106+
If using a Linux client, use the following command to mount the SMB share.
107+
108+
```
109+
sudo mount -t nfs -o vers=2.1 10.126.76.172:/databoxe2etest_BlockBlob /home/databoxubuntuhost/databox
110+
```
111+
112+
The `vers` parameter is the version of SMB that your Linux host supports. Plug in the appropriate version in the above command.
113+
114+
For versions of SMB that the Data Box Heavy supports, see [Supported file systems for Linux clients](data-box-heavy-system-requirements.md#supported-file-systems-for-linux-clients).
115+
116+
## Copy data to Data Box Heavy
117+
118+
Once you're connected to the Data Box Heavy shares, the next step is to copy data.
119+
120+
### Copy considerations
121+
122+
Before you begin the data copy, review the following considerations:
123+
124+
- Make sure that you copy the data to shares that correspond to the appropriate data format. For instance, copy the block blob data to the share for block blobs. Copy the VHDs to page blob.
125+
126+
If the data format doesn't match the appropriate share type, then at a later step, the data upload to Azure will fail.
127+
- While copying data, make sure that the data size conforms to the size limits described in the [Azure storage and Data Box Heavy limits](data-box-heavy-limits.md).
128+
- If data, which is being uploaded by Data Box Heavy, is concurrently uploaded by other applications outside of Data Box Heavy, then this could result in upload job failures and data corruption.
129+
- We recommend that:
130+
- You don't use both SMB and NFS at the same time.
131+
- Copy the same data to same end destination on Azure.
132+
133+
In these cases, the final outcome can't be determined.
134+
- Always create a folder for the files that you intend to copy under the share and then copy the files to that folder. The folder created under block blob and page blob shares represents a container to which the data is uploaded as blobs. You cannot copy files directly to *root* folder in the storage account.
135+
136+
After you've connected to the SMB share, begin data copy.
137+
138+
1. You can use any SMB compatible file copy tool such as Robocopy to copy your data. Multiple copy jobs can be initiated using Robocopy. Use the following command:
139+
140+
```
141+
robocopy <Source> <Target> * /e /r:3 /w:60 /is /nfl /ndl /np /MT:32 or 64 /fft /Log+:<LogFile>
142+
```
143+
The attributes are described in the following table.
144+
145+
|Attribute |Description |
146+
|---------|---------|
147+
|/e |Copies subdirectories including empty directories. |
148+
|/r: |Specifies the number of retries on failed copies. |
149+
|/w: |Specifies the wait time between retries, in seconds. |
150+
|/is |Includes the same files. |
151+
|/nfl |Specifies that file names aren't logged. |
152+
|/ndl |Specifies that directory names aren't logged. |
153+
|/np |Specifies that the progress of the copying operation (the number of files or directories copied so far) will not be displayed. Displaying the progress significantly lowers the performance. |
154+
|/MT | Use multithreading, recommended 32 or 64 threads. This option not used with encrypted files. You may need to separate encrypted and unencrypted files. However, single threaded copy significantly lowers the performance. |
155+
|/fft | Use to reduce the time stamp granularity for any file system. |
156+
|/b | Copies files in Backup mode. |
157+
|/z | Copies files in Restart mode, use this if the environment is unstable. This option reduces throughput due to additional logging. |
158+
| /zb | Uses Restart mode. If access is denied, this option uses Backup mode. This option reduces throughput due to checkpointing. |
159+
|/efsraw | Copies all encrypted files in EFS raw mode. Use only with encrypted files. |
160+
|log+:\<LogFile>| Appends the output to the existing log file.|
161+
162+
163+
The following sample shows the output of the robocopy command to copy files to the Data Box Heavy.
164+
165+
```
166+
C:\Users>Robocopy C:\Git\azure-docs-pr\contributor-guide \\10.100.10.100\devicemanagertest1_AzFile\templates /MT:24
167+
-------------------------------------------------------------------------------
168+
ROBOCOPY :: Robust File Copy for Windows
169+
-------------------------------------------------------------------------------
170+
Started : Thursday, April 4, 2019 2:34:58 PM
171+
Source : C:\Git\azure-docs-pr\contributor-guide\
172+
Dest : \\10.100.10.100\devicemanagertest1_AzFile\templates\
173+
Files : *.*
174+
Options : *.* /DCOPY:DA /COPY:DAT /MT:24 /R:5 /W:60
175+
------------------------------------------------------------------------------
176+
177+
100% New File 206 C:\Git\azure-docs-pr\contributor-guide\article-metadata.md
178+
100% New File 209 C:\Git\azure-docs-pr\contributor-guide\content-channel-guidance.md
179+
100% New File 732 C:\Git\azure-docs-pr\contributor-guide\contributor-guide-index.md
180+
100% New File 199 C:\Git\azure-docs-pr\contributor-guide\contributor-guide-pr-criteria.md
181+
New File 178 C:\Git\azure-docs-pr\contributor-guide\contributor-guide-pull-request-co100% .md
182+
New File 250 C:\Git\azure-docs-pr\contributor-guide\contributor-guide-pull-request-et100% e.md
183+
100% New File 174 C:\Git\azure-docs-pr\contributor-guide\create-images-markdown.md
184+
100% New File 197 C:\Git\azure-docs-pr\contributor-guide\create-links-markdown.md
185+
100% New File 184 C:\Git\azure-docs-pr\contributor-guide\create-tables-markdown.md
186+
100% New File 208 C:\Git\azure-docs-pr\contributor-guide\custom-markdown-extensions.md
187+
100% New File 210 C:\Git\azure-docs-pr\contributor-guide\file-names-and-locations.md
188+
100% New File 234 C:\Git\azure-docs-pr\contributor-guide\git-commands-for-master.md
189+
100% New File 186 C:\Git\azure-docs-pr\contributor-guide\release-branches.md
190+
100% New File 240 C:\Git\azure-docs-pr\contributor-guide\retire-or-rename-an-article.md
191+
100% New File 215 C:\Git\azure-docs-pr\contributor-guide\style-and-voice.md
192+
100% New File 212 C:\Git\azure-docs-pr\contributor-guide\syntax-highlighting-markdown.md
193+
100% New File 207 C:\Git\azure-docs-pr\contributor-guide\tools-and-setup.md
194+
------------------------------------------------------------------------------
195+
196+
Total Copied Skipped Mismatch FAILED Extras
197+
Dirs : 1 1 1 0 0 0
198+
Files : 17 17 0 0 0 0
199+
Bytes : 3.9 k 3.9 k 0 0 0 0
200+
C:\Users>
201+
```
202+
203+
2. To optimize the performance, use the following robocopy parameters when copying the data. (The numbers below represent the best case scenarios.)
204+
205+
| Platform | Mostly small files < 512 KB | Mostly medium files 512 KB-1 MB | Mostly large files > 1 MB |
206+
|-------------|--------------------------------|----------------------------|----------------------------|
207+
| Data Box Heavy | 6 Robocopy sessions <br> 24 threads per sessions | 6 Robocopy sessions <br> 16 threads per sessions | 6 Robocopy sessions <br> 16 threads per sessions |
208+
209+
210+
For more information on Robocopy command, go to [Robocopy and a few examples](https://social.technet.microsoft.com/wiki/contents/articles/1073.robocopy-and-a-few-examples.aspx).
211+
212+
3. Open the target folder to view and verify the copied files.
213+
214+
![View copied files](media/data-box-heavy-deploy-copy-data/view-copied-files-1.png)
215+
216+
217+
4. As the data is copied:
218+
219+
- The file names, sizes, and format are validated to ensure those meet the Azure object and storage limits as well as Azure file and container naming conventions.
220+
- To ensure data integrity, checksum is also computed inline.
221+
222+
If you have any errors during the copy process, download the error files for troubleshooting. Select the arrow icon to download the error files.
223+
224+
![Download error files](media/data-box-heavy-deploy-copy-data/download-error-files.png)
225+
226+
For more information, see [View error logs during data copy to Data Box Heavy](data-box-logs.md#view-error-log-during-data-copy-to-data-box). For a detailed list of errors during data copy, see [Troubleshoot Data Box Heavy issues](data-box-troubleshoot.md).
227+
228+
5. Open the error file in Notepad. The following error file indicates that the data is not aligned correctly.
229+
230+
![Open error file](media/data-box-heavy-deploy-copy-data/open-error-file.png)
231+
232+
For a page blob, the data needs to be 512 bytes aligned. After this data is removed, the error resolves as shown in the following screenshot.
233+
234+
![Error resolved](media/data-box-heavy-deploy-copy-data/error-resolved.png)
235+
236+
6. After the copy is complete, go to **View Dashboard** page. Verify the used space and the free space on your device.
237+
238+
![Verify free and used space on dashboard](media/data-box-heavy-deploy-copy-data/verify-used-space-dashboard.png)
239+
240+
Repeat the above steps to copy data on to the second node of the device.
241+
242+
## Next steps
243+
244+
In this tutorial, you learned about Azure Data Box Heavy topics such as:
245+
246+
> [!div class="checklist"]
247+
> * Connect to Data Box Heavy
248+
> * Copy data to Data Box Heavy
249+
250+
251+
Advance to the next tutorial to learn how to ship your Data Box Heavy back to Microsoft.
252+
253+
> [!div class="nextstepaction"]
254+
> [Ship your Azure Data Box Heavy to Microsoft](./data-box-heavy-deploy-picked-up.md)
255+

0 commit comments

Comments
 (0)