You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/storage/common/storage-ref-azcopy-bench.md
+43-64Lines changed: 43 additions & 64 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,120 +4,99 @@ description: This article provides reference information for the azcopy bench co
4
4
author: normesta
5
5
ms.service: storage
6
6
ms.topic: reference
7
-
ms.date: 07/24/2020
7
+
ms.date: 05/26/2022
8
8
ms.author: normesta
9
9
ms.subservice: common
10
10
ms.reviewer: zezha-msft
11
11
---
12
12
13
-
# azcopy benchmark
13
+
# azcopy bench
14
14
15
-
Runs a performance benchmark by uploading or downloading test data to or from a specified destination.
16
-
For uploads, the test data is automatically generated.
15
+
Runs a performance benchmark by uploading or downloading test data to or from a specified destination. For uploads, the test data is automatically generated.
17
16
18
17
The benchmark command runs the same process as 'copy', except that:
19
18
20
-
- Instead of requiring both source and destination parameters, benchmark takes just one. This is the
21
-
blob container, Azure Files Share, or Azure Data Lake Storage Gen2 file system that you want to upload to or download from.
19
+
- Instead of requiring both source and destination parameters, benchmark takes just one. This is the blob container, Azure Files Share, or Azure Data Lake Storage Gen2 file system that you want to upload to or download from.
22
20
23
-
- The 'mode' parameter describes whether AzCopy should test uploads to or downloads from given target. Valid values are 'Upload'
21
+
- The 'mode' parameter describes whether AzCopy should test uploads to or downloads from given target. Valid values ar`e 'Upload'
24
22
and 'Download'. Default value is 'Upload'.
25
23
26
-
- For upload benchmarks, the payload is described by command-line parameters, which control how many files are autogenerated and
27
-
how significant the files are. The generation process takes place entirely in memory. Disk is not used.
24
+
- For upload benchmarks, the payload is described by commandline parameters, which control how many files are auto-generated and
25
+
how big they are. The generation process takes place entirely in memory. Disk isn't used.
28
26
29
-
- For downloads, the payload consists of whichever files already exist at the source. (See example below about how to generate
27
+
- For downloads, the payload consists of whichever files already exist at the source. (See example below about how to generate
30
28
test files if needed).
29
+
30
+
- Only a few of the optional parameters that are available to the copy command are supported.
31
+
32
+
- Additional diagnostics are measured and reported.
33
+
34
+
- For uploads, the default behavior is to delete the transferred data at the end of the test run. For downloads, the data is never actually saved locally.
31
35
32
-
- Only a few of the optional parameters that are available to the copy command are supported.
33
-
34
-
- Additional diagnostics are measured and reported.
35
-
36
-
- For uploads, the default behavior is to delete the transferred data at the end of the test run. For downloads, the data
37
-
is never saved locally.
38
-
39
-
Benchmark mode will automatically tune itself to the number of parallel TCP connections that gives
40
-
the maximum throughput. It will display that number at the end. To prevent autotuning, set the
41
-
AZCOPY_CONCURRENCY_VALUE environment variable to a specific number of connections.
36
+
Benchmark mode will automatically tune itself to the number of parallel TCP connections that gives the maximum throughput. It will display that number at the end. To prevent auto-tuning, set the COPY_CONCURRENCY_VALUE environment variable to a specific number of connections.
42
37
43
38
All the usual authentication types are supported. However, the most convenient approach for benchmarking upload is typically
44
39
to create an empty container with a SAS token and use SAS authentication. (Download mode requires a set of test data to be
45
40
present in the target container.)
46
-
47
-
## Related conceptual articles
48
-
49
-
-[Get started with AzCopy](storage-use-azcopy-v10.md)
50
-
-[Optimize the performance of AzCopy v10 with Azure Storage](storage-use-azcopy-optimize.md)
51
-
52
-
## Examples
53
-
41
+
54
42
```azcopy
55
-
azcopy benchmark [destination] [flags]
43
+
azcopy bench [destination] [flags]
56
44
```
57
45
58
-
Run a benchmark test with default parameters (suitable for benchmarking networks up to 1 Gbps):'
Same as above, but use 50,000 files, each 8 MiB in size and compute their MD5 hashes (in the same way that the --put-md5 flag does this
57
+
in the copy command). The purpose of --put-md5 when benchmarking is to test whether MD5 computation affects throughput for the selected file count and size:
**--blob-type** string Defines the type of blob at the destination. Used to allow benchmarking different blob types. Identical to the same-named parameter in the copy command (default "Detect").
71
+
`--blob-type string` defines the type of blob at the destination. Used to allow benchmarking different blob types. Identical to the same-named parameter in the copy command (default "Detect")
93
72
94
-
**--block-size-mb** floatUse this block size (specified in MiB). Default is automatically calculated based on file size. Decimal fractions are allowed - for example, 0.25. Identical to the same-named parameter in the copy command.
73
+
`--block-size-mb float`Use this block size (specified in MiB). Default is automatically calculated based on file size. Decimal fractions are allowed - for example, 0.25. Identical to the same-named parameter in the copy command
95
74
96
-
**--check-length**Check the length of a file on the destination after the transfer. If there is a mismatch between source and destination, the transfer is marked as failed. (default true)
75
+
`--check-length`Check the length of a file on the destination after the transfer. If there's a mismatch between source and destination, the transfer is marked as failed. (default true)
97
76
98
-
**--delete-test-data**If true, the benchmark data will be deleted at the end of the benchmark run. Set it to false if you want to keep the data at the destination - for example, to use it for manual tests outside benchmark mode (default true).
77
+
`--delete-test-data`If true, the benchmark data will be deleted at the end of the benchmark run. Set it to false if you want to keep the data at the destination - for example, to use it for manual tests outside benchmark mode (default true)
99
78
100
-
**--file-count**uint. The number of autogenerated data files to use (default 100).
79
+
`--file-count` (uint) number of auto-generated data files to use (default 100)
101
80
102
-
**--help** Help for bench
81
+
`-h`, `--help` help for bench
103
82
104
-
**--log-level**string Define the log verbosity for the log file, available levels: INFO(all requests/responses), WARNING(slow responses), ERROR(only failed requests), and NONE(no output logs). (default "INFO")
83
+
`--log-level` (string) define the log verbosity for the log file, available levels: INFO(all requests/responses), WARNING(slow responses), ERROR(only failed requests), and NONE(no output logs). (default "INFO")
105
84
106
-
**--mode**stringDefines if Azcopy should test uploads or downloads from this target. Valid values are 'upload' and 'download'. Defaulted option is 'upload'. (default 'upload')
85
+
`--mode` (string) Defines if Azcopy should test uploads or downloads from this target. Valid values are 'upload' and 'download'. Defaulted option is 'upload'. (default "upload")
107
86
108
-
**--number-of-folders**uint If larger than 0, create folders to divide up the data.
87
+
`--number-of-folders` (uint) If larger than 0, create folders to divide up the data.
109
88
110
-
**--put-md5**Create an MD5 hash of each file, and save the hash as the Content-MD5 property of the destination blob/file. (By default the hash is NOT created.) Identical to the same-named parameter in the copy command.
89
+
`--put-md5`Create an MD5 hash of each file, and save the hash as the Content-MD5 property of the destination blob/file. (By default the hash is NOT created.) Identical to the same-named parameter in the copy command
111
90
112
-
**--size-per-file**string Size of each autogenerated data file. Must be a number immediately followed by K, M, or G. E.g. 12k or 200G (default "250M").
91
+
`--size-per-file` (string)Size of each auto-generated data file. Must be a number immediately followed by K, M or G. E.g. 12k or 200G (default "250M")
113
92
114
93
## Options inherited from parent commands
115
94
116
-
**--cap-mbpsfloat**Caps the transfer rate, in megabits per second. Moment-by-moment throughput might vary slightly from the cap. If this option is set to zero, or it is omitted, the throughput isn't capped.
95
+
`--cap-mbps` (float) Caps the transfer rate, in megabits per second. Moment-by-moment throughput might vary slightly from the cap. If this option is set to zero, or it's omitted, the throughput isn't capped.
117
96
118
-
**--output-type**stringFormat of the command's output. The choices include: text, json. The default value is 'text'. (default "text").
97
+
`--output-type` (string) Format of the command's output. The choices include: text, json. The default value is 'text'. (default "text")
119
98
120
-
**--trusted-microsoft-suffixes**string Specifies additional domain suffixes where Azure Active Directory login tokens may be sent. The default is '*.core.windows.net;*.core.chinacloudapi.cn;*.core.cloudapi.de;*.core.usgovcloudapi.net'. Any listed here are added to the default. For security, you should only put Microsoft Azure domains here. Separate multiple entries with semi-colons.
99
+
`--trusted-microsoft-suffixes` (string)Specifies additional domain suffixes where Azure Active Directory login tokens may be sent. The default is '*.core.windows.net;*.core.chinacloudapi.cn;*.core.cloudapi.de;*.core.usgovcloudapi.net;*.storage.azure.net'. Any listed here are added to the default. For security, you should only put Microsoft Azure domains here. Separate multiple entries with semi-colons.
0 commit comments