You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/media-services/latest/content-aware-encoding.md
+20-12Lines changed: 20 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
# Mandatory fields. See more on aka.ms/skyeye/meta.
3
-
title: A preset for content-aware encoding - Azure Media Services |
3
+
title: A preset for content-aware encoding - Azure Media Services
4
4
description: This article discusses content-aware encoding in Microsoft Azure Media Services v3.
5
5
services: media-services
6
6
documentationcenter: ''
@@ -17,19 +17,19 @@ ms.custom:
17
17
18
18
---
19
19
20
-
# A preset for content-aware encoding
20
+
# Use the content-aware encoding preset to find the optimal bitrate value for a given resolution
21
21
22
-
In order to prepare content for delivery by [adaptive bitrate streaming](https://en.wikipedia.org/wiki/Adaptive_bitrate_streaming), video needs to be encoded at multiple bit-rates (high to low). In order to ensure graceful degradation of quality, as the bitrate is lowered so is the resolution of the video. This results in a so-called encoding ladder – a table of resolutions and bitrates; see the Media Services [built-in encoding presets](https://docs.microsoft.com/rest/api/media/transforms/createorupdate#encodernamedpreset).
22
+
In order to prepare content for delivery by [adaptive bitrate streaming](https://en.wikipedia.org/wiki/Adaptive_bitrate_streaming), video needs to be encoded at multiple bit-rates (high to low). This ensures graceful degradation of quality, as the bitrate is lowered so is the resolution of the video. Such multiple bit-rate encoding makes use of a so-called encoding ladder – a table of resolutions and bitrates, see the Media Services [built-in encoding presets](https://docs.microsoft.com/rest/api/media/transforms/createorupdate#encodernamedpreset).
23
23
24
-
## Overview
24
+
You should be aware of the content you are processing, and customize/tune the encoding ladder to the complexity of the individual video. At each resolution, there is a bitrate beyond which any increase in quality is not perceptive – the encoder operates at this optimal bitrate value. The next level of optimization is to select the resolutions based on the content – for example, a video of a PowerPoint presentation does not benefit from going below 720p. Going further, the encoder can be tasked to optimize the settings for each shot within the video.
25
25
26
-
Interest in moving beyond a one-preset-fits-all-videos approach increased after Netflix published their [blog](https://medium.com/netflix-techblog/per-title-encode-optimization-7e99442b62a2) in December 2015. Since then, multiple solutions for content-aware encoding have been released in the marketplace; see [this article](https://www.streamingmedia.com/Articles/Editorial/Featured-Articles/Buyers-Guide-to-Per-Title-Encoding-130676.aspx) for an overview. The idea is to be aware of the content – to customize or tune the encoding ladder to the complexity of the individual video. At each resolution, there is a bitrate beyond which any increase in quality is not perceptive – the encoder operates at this optimal bitrate value. The next level of optimization is to select the resolutions based on the content – for example, a video of a PowerPoint presentation does not benefit from going below 720p. Going further, the encoder can be tasked to optimize the settings for each shot within the video. Netflix described [such an approach](https://medium.com/netflix-techblog/optimized-shot-based-encodes-now-streaming-4b9464204830) in 2018.
26
+
Microsoft's [Adaptive Streaming](autogen-bitrate-ladder.md) preset partially addresses the problem of the variability in the quality and resolution of the source videos. Our customers have a varying mix of content, some at 1080p, others at 720p, and a few at SD and lower resolutions. Furthermore, not all source content are high quality mezzanines from film or TV studios. The Adaptive Streaming preset addresses these problems by ensuring that the bitrate ladder never exceeds the resolution or the average bitrate of the input mezzanine. However, this preset does not examine source properties other than resolution and bitrate.
27
27
28
-
In early 2017, Microsoft released the [Adaptive Streaming](autogen-bitrate-ladder.md) preset to address the problem of the variability in the quality and resolution of the source videos. Our customers had a varying mix of content, some at 1080p, others at 720p, and a few at SD and lower resolutions. Furthermore, not all source content was high quality mezzanines from film or TV studios. The Adaptive Streaming preset addresses these problems by ensuring that the bitrate ladder never exceeds the resolution or the average bitrate of the input mezzanine.
28
+
## The content-aware encoding
29
29
30
-
The new content-aware encoding preset extends that mechanism, by incorporating custom logic that lets the encoder seek the optimal bitrate value for a given resolution, but without requiring extensive computational analysis. This preset produces a set of GOP-aligned MP4s. Given any input content, the service performs an initial lightweight analysis of the input content, and uses the results to determine the optimal number of layers, appropriate bitrate and resolution settings for delivery by adaptive streaming. This preset is particularly effective for low and medium complexity videos, where the output files will be at lower bitrates than the Adaptive Streaming preset but at a quality that still delivers a good experience to viewers. The output will contain MP4 files with video and audio interleaved
30
+
The content-aware encoding preset extends the "adaptive bitrate streaming" mechanism, by incorporating custom logic that lets the encoder seek the optimal bitrate value for a given resolution, but without requiring extensive computational analysis. This preset produces a set of GOP-aligned MP4s. Given any input content, the service performs an initial lightweight analysis of the input content, and uses the results to determine the optimal number of layers, appropriate bitrate and resolution settings for delivery by adaptive streaming. This preset is particularly effective for low and medium complexity videos, where the output files will be at lower bitrates than the Adaptive Streaming preset but at a quality that still delivers a good experience to viewers. The output will contain MP4 files with video and audio interleaved
31
31
32
-
See the following sample graphs that show the comparison using quality metrics like [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio) and [VMAF](https://en.wikipedia.org/wiki/Video_Multimethod_Assessment_Fusion). The source was created by concatenating short clips of high complexity shots from movies and TV shows, intended to stress the encoder. By definition, this preset produces results that vary from content to content – it also means that for some content, there may not be significant reduction in bitrate or improvement in quality.
32
+
The following sample graphs show the comparison using quality metrics like [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio) and [VMAF](https://en.wikipedia.org/wiki/Video_Multimethod_Assessment_Fusion). The source was created by concatenating short clips of high complexity shots from movies and TV shows, intended to stress the encoder. By definition, this preset produces results that vary from content to content – it also means that for some content, there may not be significant reduction in bitrate or improvement in quality.
33
33
34
34

35
35
@@ -49,9 +49,13 @@ Below are the results for another category of source content, where the encoder
49
49
50
50
**Figure 4: RD curve using VMAF for low quality input (at 1080p)**
51
51
52
-
## Use the content-aware encoding preset
52
+
## How to use the content-aware encoding preset
53
+
54
+
You can create transforms that use this preset as follows.
55
+
56
+
> [!TIP]
57
+
> See the [Next steps](#next-steps) section for tutorials that use tranform outputs. The output asset can be delivered from Media Services streaming endpoints in protocols such as MPEG-DASH and HLS (as shown in the tutorials).
53
58
54
-
You can create transforms that use this preset as follows. If using a tutorial [such as this](stream-files-tutorial-with-api.md), you can update the code as follows:
55
59
56
60
```csharp
57
61
TransformOutput[] output=newTransformOutput[]
@@ -70,8 +74,12 @@ TransformOutput[] output = new TransformOutput[]
70
74
```
71
75
72
76
> [!NOTE]
73
-
> The underlying algorithms are subject to further improvements. There can and will be changes over time to the logic used for generating bitrate ladders, with the goal of providing an algorithm that is robust, and adapts to a wide variety of input conditions. Encoding jobs using this preset will still be billed based on output minutes, and the output asset can be delivered from our streaming endpoints in protocols such as DASH and HLS.
77
+
> Encoding jobs using the `ContentAwareEncoding`preset are being billed based on the output minutes.
74
78
75
79
## Next steps
76
80
77
-
Now that you have learned about this new option of optimizing your videos, we invite you to try it out. You can send us feedback using the links at the end of this article.
81
+
*[Tutorial: Upload, encode, and stream videos with Media Services v3](stream-files-tutorial-with-api.md)
82
+
*[Tutorial: Encode a remote file based on URL and stream the video - REST](stream-files-tutorial-with-rest.md)
83
+
*[Tutorial: Encode a remote file based on URL and stream the video - CLI](stream-files-cli-quickstart.md)
84
+
*[Tutorial: Encode a remote file based on URL and stream the video - .NET](stream-files-dotnet-quickstart.md)
85
+
*[Tutorial: Encode a remote file based on URL and stream the video - Node.js](stream-files-nodejs-quickstart.md)
0 commit comments