Skip to content

Commit 3d72bdf

Browse files
Stacyrch140Stacyrch140
authored
Merge pull request #245027 from chengyuanlai-msft/patch-1
Create Quickstart for Video Constraints
2 parents 978dc3a + 595cd27 commit 3d72bdf

File tree

7 files changed

+518
-89
lines changed

7 files changed

+518
-89
lines changed

‎articles/communication-services/concepts/voice-video-calling/video-constraints.md

Lines changed: 10 additions & 89 deletions
Original file line numberDiff line numberDiff line change
@@ -17,101 +17,22 @@ ms.subservice: calling
1717

1818
[!INCLUDE [Public Preview Disclaimer](../../includes/public-preview-include.md)]
1919

20-
The Video Constraints API is a powerful tool that enables developers to control the video quality from within their video calls. With this API, developers can set constraints on the video resolution to ensure that the video call is optimized for the user's device and network conditions. The ACS video engine is optimized to allow the video quality to change dynamically based on devices ability and network quality. But there might be certain scenarios where you would want to have tighter control of the video quality that end users will experience. For example, there might be cases when pushing the highest video quality isn't a top priority or you may want to limit the video bandwidth usage in the application. To support this, you can use the Video Constraints API to have tighter control video quality.
20+
The Video Constraints API is a powerful tool that enables developers to control the video quality from within their video calls. With this API, developers can set constraints on the video resolution to ensure that the video call is optimized for the user's device and network conditions. The ACS video engine is optimized to allow the video quality to change dynamically based on devices ability and network quality. But there might be certain scenarios where you would want to have tighter control of the video quality that end users experience. For instance, there may be situations where the highest video quality is not a priority or you may want to limit the video bandwidth usage in the application. To support those use cases, you can use the Video Constraints API to have tighter control over video quality.
2121

2222
Another benefit of the Video Constraints API is that it enables developers to optimize the video call for different devices. For example, if a user is using an older device with limited processing power, developers can set constraints on the video resolution to ensure that the video call runs smoothly on that device
2323

24-
Currently ACS supports setting the maximum video resolution that a client will send. The maximum resolution is set at the start of the call and is static throughout the entire call. The sender max video resolution constraint is supported on Desktop browsers (Chrome, Edge, Firefox) and when using iOS Safari mobile browser.
24+
ACS Calling SDK (Web) currently supports setting the maximum video resolution that a client sends. The maximum resolution is set at the start of the call and is static throughout the entire call. The sender max video resolution constraint is supported on Desktop browsers (Chrome, Edge, Firefox) and when using iOS Safari mobile browser.
2525

26-
> [!NOTE]
27-
> Video Constraints is currently supported only for our JavaScript / Web SDK, and is available starting in version [1.11.0-beta.1](https://www.npmjs.com/package/@azure/communication-calling/v/1.11.0-beta.1) of the Calling SDK.
26+
ACS Calling SDK (Android/iOS/Windows) currently supports setting the maximum values of video resolution and framerate for outgoing video streams and setting the maximum resolution for incoming video streams. The constraints can be set at the start of the call and during the call.
2827

29-
> [!NOTE]
30-
> Future versions of the Video Contraint API will allow enhanced ability by giving control to set the maximum FPS and bitrate, as well as the ability to enforce the video constraint at different points of a call (and not just at the start of a video call).
28+
## Supported constraints
3129

32-
## Using video constraints
33-
The video constraints setting is implemented on the `Call` interface. To use the Video Constraints, you can specify the constraints from within CallOptions when you make a call, accept a call, or join a call. You will also have to specify `localVideoStreams` in `videoOptions`. Do note that constraints don't work if you join a call with audio only option and turn on the camera later.
34-
35-
```javascript
36-
const callOptions = {
37-
videoOptions: {
38-
localVideoStreams: [...],
39-
constraints: {
40-
send: {
41-
height: {
42-
max: 240
43-
}
44-
}
45-
}
46-
},
47-
audioOptions: {
48-
muted: false
49-
}
50-
};
51-
// make a call
52-
this.callAgent.startCall(identitiesToCall, callOptions);
53-
// join a group call
54-
this.callAgent.join({ groupId }, callOptions);
55-
// accept an incoming call
56-
this.incomingCall.accept(callOptions)
57-
```
58-
59-
Video constraints types are described as follows:
60-
```javascript
61-
export declare interface VideoOptions {
62-
localVideoStreams?: LocalVideoStream[];
63-
//video constraint when call starts
64-
constraints?: VideoConstraints;
65-
};
66-
export declare type VideoConstraints = {
67-
send?: VideoSendConstraints;
68-
};
69-
export declare type VideoSendConstraints = {
70-
height?: MediaConstraintRange;
71-
};
72-
export declare type MediaConstraintRange = {
73-
max?: number;
74-
};
75-
```
76-
77-
### Sender max video resolution constraint
78-
79-
With sender max video resolution constraint, you can limit the max video resolution of the sending stream. The value you have to provide for this constraint is `height`.
80-
81-
```javascript
82-
{
83-
localVideoStreams: [...],
84-
constraints: {
85-
send: {
86-
height: {
87-
max: 240
88-
}
89-
}
90-
}
91-
}
92-
```
93-
When setting the maximum resolution video constraints, the SDK will choose the nearest resolution that falls within the constraint set (to prevent the resolution height doesn't exceed the maximum constraint value allowed). Also, when the resolution constraint value is too small, the SDK will choose the smallest available resolution. In this case, the height of chosen resolution can be larger than the constraint value.
94-
95-
> [!NOTE]
96-
> The resolution constraint is a `max` constraint, which means the possible resolutions can be the specified resolution or smaller.
97-
> There is no gurantee that the sent video resolution will remain at the specified resolution.
98-
99-
The `height` in `VideoSendConstraints` has a different meaning when a mobile device is in portrait mode. In portrait mode, this value indicates the shorter side of the device. For example, specifying `constraints.send.height.max` value with 240 on a 1080(W) x 1920(H) device in portrait mode, the constraint height is on the 1080(W) side. When the same device is in landscape mode (1920(W) x 1080(H)), the constraint is on the 1080(H) side.
100-
101-
If you use MediaStats API to track the sent video resolution, you may find out that the sent resolution can change during the call. It can go up and down, but should be equal or smaller than the constraint value you provide. This resolution change is an expected behavior. The browser also has some degradation rule to adjust sent resolution based on cpu or network conditions.
102-
103-
### Media stats
104-
To evaluate and compare the video quality after applying the video constraints, you can access [MediaStats API](./media-quality-sdk.md) to get video resolution and bitrate information of the sending stream. The media stats also include other granular stats related to the streams, such as jitter, packet loss, round trip time, etc.
105-
106-
```javascript
107-
const mediaStatsFeature = call.feature(Features.MediaStats);
108-
const mediaStatsCollector = mediaStatsFeature.createCollector();
109-
110-
mediaStatsCollector.on('sampleReported', (sample: SDK.MediaStatsReportSample) => {
111-
// process the stats for the call.
112-
console.log(sample);
113-
});
114-
```
30+
| Platform | Supported Constraints |
31+
| ----------- | ----------- |
32+
| Web | outgoing video: resolution |
33+
| Android | incoming video: resolution, outgoing video: resolution \| framerate |
34+
| iOS | incoming video: resolution, outgoing video: resolution \| framerate |
35+
| Windows | incoming video: resolution, outgoing video: resolution \| framerate |
11536

11637
## Next steps
11738
For more information, see the following articles:

‎articles/communication-services/quickstarts/voice-video-calling/get-started-video-constraints.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
ms.author: chengyuanlai
3+
title: Quickstart - Set video constraints in your calling app
4+
titleSuffix: An Azure Communication Services quickstart
5+
description: In this quickstart, you learn how to set video constraints in your existing calling app using Azure Communication Services.
6+
author: sloanster
7+
services: azure-communication-services
8+
ms.date: 07/13/2023
9+
ms.topic: quickstart
10+
ms.service: azure-communication-services
11+
ms.subservice: calling
12+
zone_pivot_groups: acs-plat-web-ios-android-windows
13+
ms.custom: mode-other, devx-track-js
14+
---
15+
16+
# Quickstart: Set video constraints in your calling app
17+
18+
::: zone pivot="platform-windows"
19+
[!INCLUDE [Set video constraints with Windows](./includes/video-constraints/video-constraints-windows.md)]
20+
::: zone-end
21+
22+
::: zone pivot="platform-android"
23+
[!INCLUDE [Set video constraints with Android](./includes/video-constraints/video-constraints-android.md)]
24+
::: zone-end
25+
26+
::: zone pivot="platform-ios"
27+
[!INCLUDE [Set video constraints with iOS](./includes/video-constraints/video-constraints-ios.md)]
28+
::: zone-end
29+
30+
::: zone pivot="platform-web"
31+
[!INCLUDE [Set video constraints with JavaScript](./includes/video-constraints/video-constraints-javascript.md)]
32+
::: zone-end
33+
## Next steps
34+
35+
For more information, see the following articles:
36+
37+
- Learn about [Video Constraints concept document](../../concepts/voice-video-calling/video-constraints.md)
38+
- Learn more about [Calling SDK capabilities](./getting-started-with-calling.md)

‎articles/communication-services/quickstarts/voice-video-calling/includes/video-constraints/video-constraints-android.md

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
---
2+
ms.author: chengyuanlai
3+
title: Quickstart - Set video constraints in your Android calling app
4+
titleSuffix: An Azure Communication Services document
5+
description: In this quickstart, you learn how to set video constraints in your existing Android calling app using Azure Communication Services.
6+
author: sloanster
7+
services: azure-communication-services
8+
ms.date: 07/10/2023
9+
ms.topic: include
10+
ms.service: azure-communication-services
11+
ms.subservice: calling
12+
---
13+
14+
[!INCLUDE [Public Preview](../../../../includes/public-preview-include-document.md)]
15+
16+
## Overview
17+
The Video Constraints API enables developers to control the video quality from within their video calls. In this quickstart guide, we illustrate how to use the API to set the constraints.
18+
19+
### Prerequisites
20+
Refer to the [Voice Calling Quickstart](../../getting-started-with-calling.md?pivots=platform-android) to set up a sample app with voice calling.
21+
22+
### Classes
23+
| Name | Description |
24+
| - | - |
25+
| VideoConstraints | Used to hold both incoming video constraints and outgoing video constraints. |
26+
| OutgoingVideoConstraints | Used to specify constraints (`maxWidth | maxHeight | maxFrameRate`) for outgoing video streams. |
27+
| IncomingVideoConstraints | Used to specify constraints (`maxWidth | maxHeight`) for incoming video streams. |
28+
29+
### Using video constraints
30+
31+
The following sections explain how the video constraints can be set for incoming and/or outgoing video streams at different times of a call.
32+
33+
#### Set video constraints before starting a call
34+
35+
For *incoming* video streams, an `IncomingVideoConstraints` needs to be added to the `IncomingVideoOptions`.
36+
```java
37+
IncomingVideoConstraints incomingVideoConstraints = new IncomingVideoConstraints();
38+
incomingVideoConstraints.setMaxWidth(/*value*/);
39+
incomingVideoConstraints.setMaxHeight(/*value*/);
40+
41+
// ...
42+
43+
IncomingVideoOptions incomingVideoOptions = new IncomingVideoOptions();
44+
incomingVideoOptions.setConstraints(incomingVideoConstraints);
45+
```
46+
47+
For *outgoing* video streams, an `OutgoingVideoConstraints` needs to be added to the `OutgoingVideoOptions`.
48+
```java
49+
OutgoingVideoConstraints outgoingVideoConstraints = new OutgoingVideoConstraints()
50+
outgoingVideoConstraints.setMaxWidth(/*value*/);
51+
outgoingVideoConstraints.setMaxHeight(/*value*/);
52+
outgoingVideoConstraints.setMaxFrameRate(/*value*/);
53+
54+
// ...
55+
56+
OutgoingVideoOptions outgoingVideoOptions = new OutgoingVideoOptions();
57+
outgoingVideoOptions.setConstraints(outgoingVideoConstraints);
58+
```
59+
60+
Since the options are used to start/join a call, the constraints can then be applied to the streams automatically. For example:
61+
```java
62+
JoinCallOptions joinCallOptions = new JoinCallOptions();
63+
joinCallOptions.setIncomingVideoOptions(incomingVideoOptions);
64+
joinCallOptions.setOutgoingVideoOptions(outgoingVideoOptions);
65+
callAgent.Join(context, locator, joinCallOptions);
66+
```
67+
68+
#### Set video constraints during a call
69+
Instead of setting the video constraints before starting a call, you can also dynamically adjust the video constraints during a call. You need to call `setVideoConstraints` on your `Call` type class and provide the constraints.
70+
```java
71+
72+
OutgoingVideoConstraints outgoingVideoConstraints = new OutgoingVideoConstraints();
73+
outgoingVideoConstraints.setMaxWidth(/*value*/);
74+
outgoingVideoConstraints.setMaxHeight(/*value*/);
75+
outgoingVideoConstraints.setMaxFrameRate(/*value*/);
76+
77+
IncomingVideoConstraints incomingVideoConstraints = new IncomingVideoConstraints();
78+
incomingVideoConstraints.setMaxWidth(/*value*/);
79+
incomingVideoConstraints.setMaxHeight(/*value*/);
80+
81+
VideoConstraints constraints = new VideoConstraints();
82+
constraints.setOutgoingVideoConstraints(outgoingVideoConstraints);
83+
constraints.setIncomingVideoConstraints(incomingVideoConstraints);
84+
85+
call.setVideoConstraints(constraints);
86+
```
87+
88+
To reset/remove the video constraints you previously set, you have to follow the above pattern and provide `0` as a constraint value. Providing `null` values for either `IncomingVideoConstraints` or `OutgoingVideoConstraints` won't reset/remove the constraints and the constraints with a `null` value will be ignored.
89+
90+
### Limitations
91+
92+
> [!NOTE]
93+
> Please make sure you are aware of these limitations when using the Video Constraints API.
94+
> Some of the limitations will be removed in future releases.
95+
96+
There are some known limitations to the current Video Constraints API.
97+
98+
* The constraint is a **max** constraint, which means the possible constraint value can be the specified value or smaller. There's no guarantee that the actual value remains the same as user-specified.
99+
100+
* When the user sets a constraint value that is too small, the SDK will use the smallest available value that is supported.
101+
102+
* For setting `OutgoingVideoConstraints` during a call, the current ongoing video stream doesn't automatically pick up the constraints specified. In order to make the constraints take effect, you need to stop and restart the outgoing video.
103+
104+
* `IncomingVideoConstraints` currently is a user-preferred constraint instead of a hard constraint, which means that depending on your network and hardware, the actual value received may still exceed the constraint set.
105+
106+
### Media stats
107+
To evaluate and compare the video quality after applying the video constraints, you can access [MediaStats API](../../../../concepts/voice-video-calling/media-quality-sdk.md) to get video resolution and bitrate information of the stream. The media stats also include other granular stats related to the streams, such as jitter, packet loss, round trip time, etc.

0 commit comments

Comments
 (0)