You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cdn/cdn-large-file-optimization.md
+22-25Lines changed: 22 additions & 25 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,53 +2,50 @@
2
2
title: Large file download optimization with Azure CDN
3
3
description: Learn how large file downloads can be optimized in Azure Content Delivery Network. This article includes several scenarios.
4
4
services: cdn
5
-
documentationcenter: ''
6
5
author: duongau
7
-
manager: danielgi
8
-
editor: ''
9
-
10
-
ms.assetid:
6
+
manager: kumudd
11
7
ms.service: azure-cdn
12
8
ms.workload: tbd
13
9
ms.tgt_pltfrm: na
14
10
ms.topic: how-to
15
-
ms.date: 05/01/2018
11
+
ms.date: 02/27/2023
16
12
ms.author: duau
17
13
---
14
+
18
15
# Large file download optimization with Azure CDN
19
16
20
-
File sizes of content delivered over the internet continue to grow due to enhanced functionality, improved graphics, and rich media content. This growth is driven by many factors: broadband penetration, larger inexpensive storage devices, widespread increase of high-definition video, and internet-connected devices (IoT). A fast and efficient delivery mechanism for large files is critical to ensure a smooth and enjoyable consumer experience.
17
+
File sizes of content delivered over the internet continue to grow due to enhanced functionality, improved graphics, and rich media content. This growth gets driven by many factors: broadband penetration, larger inexpensive storage devices, widespread increase of high-definition video, and internet-connected devices (IoT). A fast and efficient delivery mechanism for large files is critical to ensure a smooth and enjoyable consumer experience.
21
18
22
19
Delivery of large files has several challenges. First, the average time to download a large file can be significant because applications might not download all data sequentially. In some cases, applications might download the last part of a file before the first part. When only a small amount of a file is requested or a user pauses a download, the download can fail. The download also might be delayed until after the content delivery network (CDN) retrieves the entire file from the origin server.
23
20
24
-
Second, the latency between a user's machine and the file determines the speed at which they can view content. In addition, network congestion and capacity problems also affect throughput. Greater distances between servers and users create additional opportunities for packet loss to occur, which reduces quality. The reduction in quality caused by limited throughput and increased packet loss might increase the wait time for a file download to finish.
21
+
Second, the latency between a user's machine and the file determines the speed at which they can view content. In addition, network congestion and capacity problems also affect throughput. Greater distances between servers and users create more opportunities for packet loss to occur, which reduces quality. The reduction in quality caused by limited throughput and increased packet loss might increase the wait time for a file download to finish.
25
22
26
-
Third, many large files are not delivered in their entirety. Users might cancel a download halfway through or watch only the first few minutes of a long MP4 video. Therefore, software and media delivery companies want to deliver only the portion of a file that's requested. Efficient distribution of the requested portions reduces the egress traffic from the origin server. Efficient distribution also reduces the memory and I/O pressure on the origin server.
23
+
Third, many large files aren't delivered in their entirety. Users might cancel a download halfway through or watch only the first few minutes of a long MP4 video. Therefore, software and media delivery companies want to deliver only the portion of a file that's requested. Efficient distribution of the requested portions reduces the egress traffic from the origin server. Efficient distribution also reduces the memory and I/O pressure on the origin server.
27
24
28
25
29
26
## Optimize for delivery of large files with Azure CDN from Microsoft
30
27
31
-
**Azure CDN Standard from Microsoft** endpoints deliver large files without a cap on file size. Additional features are turned on by default to make delivery of large files faster.
28
+
**Azure CDN Standard from Microsoft** endpoints deliver large files without a cap on file size. Extra features are turned on by default to make delivery of large files faster.
32
29
33
30
### Object chunking
34
31
35
-
**Azure CDN Standard from Microsoft** uses a technique called object chunking. When a large file is requested, the CDN retrieves smaller pieces of the file from the origin. After the CDN POP server receives a full or byte-range file request, the CDN edge server requests the file from the origin in chunks of 8 MB.
32
+
**Azure CDN Standard from Microsoft** uses a technique called object chunking. When a large file is requested, the CDN retrieves smaller pieces of the file from the origin. After the CDN POP server receives a full or byte-ranges file request, the CDN edge server requests the file from the origin in chunks of 8 MB.
36
33
37
-
After the chunk arrives at the CDN edge, it's cached and immediately served to the user. The CDN then prefetches the next chunk in parallel. This prefetch ensures that the content stays one chunk ahead of the user, which reduces latency. This process continues until the entire file is downloaded (if requested), all byte ranges are available (if requested), or the client terminates the connection.
34
+
After the chunk arrives at the CDN edge, it's cached and immediately served to the user. The CDN then prefetches the next chunk in parallel. This prefetch ensures that the content stays one chunk ahead of the user, which reduces latency. This process continues until the entire file gets downloaded (if requested), all byte ranges are available (if requested), or the client terminates the connection.
38
35
39
36
For more information on the byte-range request, see [RFC 7233](https://tools.ietf.org/html/rfc7233).
40
37
41
-
The CDN caches any chunks as they're received. The entire file doesn't need to be cached on the CDN cache. Subsequent requests for the file or byte ranges are served from the CDN cache. If not all the chunks are cached on the CDN, prefetch is used to request chunks from the origin. This optimization relies on the ability of the origin server to support byte-range requests; if the origin server doesn't support byte-range requests, requests to download data greater than 8mb size will fail.
38
+
The CDN caches any chunks as they're received. The entire file doesn't need to be cached on the CDN cache. Subsequent requests for the file or byte ranges are served from the CDN cache. If not all the chunks are cached on the CDN, prefetch is used to request chunks from the origin. This optimization relies on the ability of the origin server to support byte-range requests. If the origin server doesn't support byte-range requests, requests to download data greater than 8-MB size fails.
42
39
43
40
### Conditions for large file optimization
44
41
There are no limits on maximum file size.
45
42
46
43
### Chunked Transfer Encoding Support
47
-
Microsoft CDN supports transfer encoding response but only up to a maximum of 8MB content size. For chunked tranfer encoded response that greater then 8MB, Microsoft CDN will only cache and serve the first 8MB of content.
44
+
Microsoft CDN supports transfer encoding response but only up to a maximum of 8-MB content size. For chunked transfer encoded response that greater than 8 MB, Microsoft CDN only cache and serve the first 8 MB of content.
48
45
49
46
## Optimize for delivery of large files with Azure CDN from Verizon
50
47
51
-
**Azure CDN Standard from Verizon** and **Azure CDN Premium from Verizon** endpoints deliver large files without a cap on file size. Additional features are turned on by default to make delivery of large files faster.
48
+
**Azure CDN Standard from Verizon** and **Azure CDN Premium from Verizon** endpoints deliver large files without a cap on file size. More features are turned on by default to make delivery of large files faster.
52
49
53
50
### Complete cache fill
54
51
@@ -60,7 +57,7 @@ The default behavior can be disabled through the rules engine in **Azure CDN Pre
60
57
61
58
### Peer cache fill hot-filing
62
59
63
-
The default peer cache fill hot-filing feature uses a sophisticated proprietary algorithm. It uses additional edge caching servers based on bandwidth and aggregate requests metrics to fulfill client requests for large, highly popular objects. This feature prevents a situation in which large numbers of extra requests are sent to a user's origin server.
60
+
The default peer cache fills hot-filing feature uses a sophisticated proprietary algorithm. It uses extra edge caching servers based on bandwidth and aggregate requests metrics to fulfill client requests for large, highly popular objects. This feature prevents a situation in which large numbers of extra requests are sent to a user's origin server.
64
61
65
62
### Conditions for large file optimization
66
63
@@ -77,7 +74,7 @@ Large file optimization is effective when certain conditions are satisfied. Cond
77
74
78
75
### Configure an Akamai CDN endpoint to optimize delivery of large files
79
76
80
-
You can configure your **Azure CDN Standard from Akamai** endpoint to optimize delivery for large files via the Azure portal. You can also use the REST APIs or any of the client SDKs to do this. The following steps show the process via the Azure portal for an **Azure CDN Standard from Akamai** profile:
77
+
You can configure your **Azure CDN Standard from Akamai** endpoint to optimize delivery for large files via the Azure portal. You can also use the REST APIs or any of the client SDKs to complete this task. The following steps show the process via the Azure portal for an **Azure CDN Standard from Akamai** profile:
81
78
82
79
1. To add a new endpoint, on an Akamai **CDN profile** page, select **Endpoint**.
83
80
@@ -92,9 +89,9 @@ After you create the CDN endpoint, it applies the large file optimizations for a
92
89
93
90
### Object chunking
94
91
95
-
Large file optimization with **Azure CDN Standard from Akamai** uses a technique called object chunking. When a large file is requested, the CDN retrieves smaller pieces of the file from the origin. After the CDN POP server receives a full or byte-range file request, it checks whether the file type is supported for this optimization. It also checks whether the file type meets the file size requirements. If the file size is greater than 10 MB, the CDN edge server requests the file from the origin in chunks of 2 MB.
92
+
Large file optimization with **Azure CDN Standard from Akamai** uses a technique called object chunking. When a large file is requested, the CDN retrieves smaller pieces of the file from the origin. After the CDN POP server receives a full or byte-rangess file request, it checks whether the file type is supported for this optimization. It also checks whether the file type meets the file size requirements. If the file size is greater than 10 MB, the CDN edge server requests the file from the origin in chunks of 2 MB.
96
93
97
-
After the chunk arrives at the CDN edge, it's cached and immediately served to the user. The CDN then prefetches the next chunk in parallel. This prefetch ensures that the content stays one chunk ahead of the user, which reduces latency. This process continues until the entire file is downloaded (if requested), all byte ranges are available (if requested), or the client terminates the connection.
94
+
After the chunk arrives at the CDN edge, it's cached and immediately served to the user. The CDN then prefetches the next chunk in parallel. This prefetch ensures that the content stays one chunk ahead of the user, which reduces latency. This process continues until the entire file gets downloaded (if requested), all byte ranges are available (if requested), or the client terminates the connection.
98
95
99
96
For more information on the byte-range request, see [RFC 7233](https://tools.ietf.org/html/rfc7233).
100
97
@@ -105,14 +102,14 @@ Large file optimization uses different default caching-expiration times from gen
105
102
106
103
| Caching | General web | Large file optimization
107
104
--- | --- | ---
108
-
Caching: Positive <br> HTTP 200, 203, 300, <br> 301, 302, and 410 | 7 days |1 day
105
+
Caching: Positive <br> HTTP 200, 203, 300, <br> 301, 302, and 410 | Seven days |One day
109
106
Caching: Negative <br> HTTP 204, 305, 404, <br> and 405 | None | 1 second
110
107
111
108
### Deal with origin failure
112
109
113
110
The origin read-timeout length increases from two seconds for general web delivery to two minutes for the large file optimization type. This increase accounts for the larger file sizes to avoid a premature timeout connection.
114
111
115
-
When a connection times out, the CDN retries a number of times before it sends a "504 - Gateway Timeout" error to the client.
112
+
When a connection times out, the CDN retries many times before it sends a "504 - Gateway Timeout" error to the client.
116
113
117
114
### Conditions for large file optimization
118
115
@@ -125,15 +122,15 @@ Minimum file size | 10 MB
125
122
Maximum file size | 150 GB
126
123
Origin server characteristics | Must support byte-range requests
127
124
128
-
## Additional considerations
125
+
## Other considerations
129
126
130
-
Consider the following additional aspects for this optimization type:
127
+
Consider the following aspects for this optimization type:
131
128
132
-
- The chunking process generates additional requests to the origin server. However, the overall volume of data delivered from the origin is much smaller. Chunking results in better caching characteristics at the CDN.
129
+
- The chunking process generates more requests to the origin server. However, the overall volume of data delivered from the origin is smaller. Chunking results in better caching characteristics at the CDN.
133
130
134
131
- Memory and I/O pressure are reduced at the origin because smaller pieces of the file are delivered.
135
132
136
-
- For chunks cached at the CDN, there are no additional requests to the origin until the content expires or it's evicted from the cache.
133
+
- For chunks cached at the CDN, there are no other requests to the origin until the content expires or it's evicted from the cache.
137
134
138
135
- Users can make range requests to the CDN, which are treated like any normal file. Optimization applies only if it's a valid file type and the byte range is between 10 MB and 150 GB. If the average file size requested is smaller than 10 MB, use general web delivery instead.
0 commit comments