Skip to content

Commit cff3671

Browse files
JamesJBarnettJamesJBarnett
authored
Merge pull request #227990 from enricohuang/patch-2
Add Video Constraints concept document
2 parents 3b2bc7b + ef8b375 commit cff3671

File tree

2 files changed

+122
-0
lines changed

2 files changed

+122
-0
lines changed

‎articles/communication-services/concepts/voice-video-calling/video-constraints.md

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
---
2+
title: Azure Communication Services Video constraints
3+
titleSuffix: An Azure Communication Services concept document
4+
description: Overview of Video Constraints
5+
author: sloanster
6+
ms.author: micahvivion
7+
manager: nmurav
8+
9+
services: azure-communication-services
10+
ms.date: 2/20/2023
11+
ms.topic: conceptual
12+
ms.service: azure-communication-services
13+
ms.subservice: calling
14+
---
15+
16+
# Video constraints
17+
18+
[!INCLUDE [Public Preview Disclaimer](../../includes/public-preview-include.md)]
19+
20+
The Video Constraints API is a powerful tool that enables developers to control the video quality from within their video calls. With this API, developers can set constraints on the video resolution to ensure that the video call is optimized for the user's device and network conditions. The ACS video engine is optimized to allow the video quality to change dynamically based on devices ability and network quality. But there might be certain scenarios where you would want to have tighter control of the video quality that end users will experience. For example, there might be cases when pushing the highest video quality isn't a top priority or you may want to limit the video bandwidth usage in the application. To support this, you can use the Video Constraints API to have tighter control video quality.
21+
22+
Another benefit of the Video Constraints API is that it enables developers to optimize the video call for different devices. For example, if a user is using an older device with limited processing power, developers can set constraints on the video resolution to ensure that the video call runs smoothly on that device
23+
24+
Currently ACS supports setting the maximum video resolution that a client will send. The maximum resolution is set at the start of the call and is static throughout the entire call. The sender max video resolution constraint is supported on Desktop browsers (Chrome, Edge, Firefox) and when using iOS Safari mobile browser.
25+
26+
> [!NOTE]
27+
> Video Constraints is currently supported only for our JavaScript / Web SDK, and is available starting in version [1.11.0-beta.1](https://www.npmjs.com/package/@azure/communication-calling/v/1.11.0-beta.1) of the Calling SDK.
28+
29+
> [!NOTE]
30+
> Future versions of the Video Contraint API will allow enhanced ability by giving control to set the maximum FPS and bitrate, as well as the ability to enforce the video constraint at different points of a call (and not just at the start of a video call).
31+
32+
## Using video constraints
33+
The video constraints setting is implemented on the `Call` interface. To use the Video Constraints, you can specify the constraints from within CallOptions when you make a call, accept a call, or join a call. You will also have to specify `localVideoStreams` in `videoOptions`. Do note that constraints don't work if you join a call with audio only option and turn on the camera later.
34+
35+
```javascript
36+
const callOptions = {
37+
videoOptions: {
38+
localVideoStreams: [...],
39+
constraints: {
40+
send: {
41+
height: {
42+
max: 240
43+
}
44+
}
45+
}
46+
},
47+
audioOptions: {
48+
muted: false
49+
}
50+
};
51+
// make a call
52+
this.callAgent.startCall(identitiesToCall, callOptions);
53+
// join a group call
54+
this.callAgent.join({ groupId }, callOptions);
55+
// accept an incoming call
56+
this.incomingCall.accept(callOptions)
57+
```
58+
59+
Video constraints types are described as follows:
60+
```javascript
61+
export declare interface VideoOptions {
62+
localVideoStreams?: LocalVideoStream[];
63+
//video constraint when call starts
64+
constraints?: VideoConstraints;
65+
};
66+
export declare type VideoConstraints = {
67+
send?: VideoSendConstraints;
68+
};
69+
export declare type VideoSendConstraints = {
70+
height?: MediaConstraintRange;
71+
};
72+
export declare type MediaConstraintRange = {
73+
max?: number;
74+
};
75+
```
76+
77+
### Sender max video resolution constraint
78+
79+
With sender max video resolution constraint, you can limit the max video resolution of the sending stream. The value you have to provide for this constraint is `height`.
80+
81+
```javascript
82+
{
83+
localVideoStreams: [...],
84+
constraints: {
85+
send: {
86+
height: {
87+
max: 240
88+
}
89+
}
90+
}
91+
}
92+
```
93+
When setting the maximum resolution video constraints, the SDK will choose the nearest resolution that falls within the constraint set (to prevent the resolution height doesn't exceed the maximum constraint value allowed). Also, when the resolution constraint value is too small, the SDK will choose the smallest available resolution. In this case, the height of chosen resolution can be larger than the constraint value.
94+
95+
> [!NOTE]
96+
> The resolution constraint is a `max` constraint, which means the possible resolutions can be the specified resolution or smaller.
97+
> There is no gurantee that the sent video resolution will remain at the specified resolution.
98+
99+
The `height` in `VideoSendConstraints` has a different meaning when a mobile device is in portrait mode. In portrait mode, this value indicates the shorter side of the device. For example, specifying `constraints.send.height.max` value with 240 on a 1080(W) x 1920(H) device in portrait mode, the constraint height is on the 1080(W) side. When the same device is in landscape mode (1920(W) x 1080(H)), the constraint is on the 1080(H) side.
100+
101+
If you use MediaStats API to track the sent video resolution, you may find out that the sent resolution can change during the call. It can go up and down, but should be equal or smaller than the constraint value you provide. This resolution change is an expected behavior. The browser also has some degradation rule to adjust sent resolution based on cpu or network conditions.
102+
103+
### Media stats
104+
To evaluate and compare the video quality after applying the video constraints, you can access [MediaStats API](./media-quality-sdk.md) to get video resolution and bitrate information of the sending stream. The media stats also include other granular stats related to the streams, such as jitter, packet loss, round trip time, etc.
105+
106+
```javascript
107+
const mediaStatsFeature = call.feature(Features.MediaStats);
108+
const mediaStatsCollector = mediaStatsFeature.createCollector();
109+
110+
mediaStatsCollector.on('sampleReported', (sample: SDK.MediaStatsReportSample) => {
111+
// process the stats for the call.
112+
console.log(sample);
113+
});
114+
```
115+
116+
## Next steps
117+
For more information, see the following articles:
118+
119+
- [Enable Media Quality Statistics in your application](./media-quality-sdk.md)
120+
- Learn about [Calling SDK capabilities](../../quickstarts/voice-video-calling/getting-started-with-calling.md)

‎articles/communication-services/toc.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,8 @@ items:
243243
href: concepts/voice-video-calling/user-facing-diagnostics.md
244244
- name: Media quality statistics
245245
href: concepts/voice-video-calling/media-quality-sdk.md
246+
- name: Video constraints
247+
href: concepts/voice-video-calling/video-constraints.md
246248
- name: Customizing the video background
247249
href: concepts/voice-video-calling/video-effects.md
248250
- name: Pre-Call diagnostics

0 commit comments

Comments
 (0)