Skip to content

Commit 1bd1695

Browse files
AnnaMHuffAnnaMHuff
authored
Merge pull request #244553 from jsaurezlee-msft/jsaurezlee-msft-patch-5
Native Media Stats Call Feature
2 parents 03321e2 + 2fa63af commit 1bd1695

File tree

6 files changed

+656
-246
lines changed

6 files changed

+656
-246
lines changed

‎articles/communication-services/concepts/voice-video-calling/includes/media-stats/media-stats-android.md

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
---
2+
title: Azure Communication Services Media Stats (Android)
3+
titleSuffix: An Azure Communication Services concept document
4+
description: Provides usage samples of the Media Stats feature Android Native.
5+
author: jsaurezle-msft
6+
ms.author: jsaurezlee
7+
8+
services: azure-communication-services
9+
ms.date: 08/09/2023
10+
ms.topic: include
11+
ms.service: azure-communication-services
12+
ms.subservice: calling
13+
---
14+
15+
## Media quality statistics for ongoing call
16+
17+
[!INCLUDE [public-preview-notes](../../../../includes/public-preview-include.md)]
18+
19+
Media quality statistics is an extended feature of the core `Call` API. You first need to obtain the MediaStats feature API object:
20+
21+
```java
22+
MediaStatsCallFeature mediaStatsCallFeature = call.feature(Features.MEDIA_STATS);
23+
```
24+
25+
The Media Stats feature object have the following API structure:
26+
- `OnSampleReportedListener`: Event for listening for periodic reports of the Media Stats.
27+
- `setSampleIntervalInSeconds(int value)`: Sets the interval in seconds of the Media Stats report generation. If not specified, sdk use defaults.
28+
- A `MediaStatsReport` object that contains the definition of the Outgoing and Incoming Media Stats categorized by Audio, Video and Screen Share.
29+
- `getOutgoingMediaStats()`: The list of Media Stats for Outgoing media.
30+
- `getAudio()`: The list of Media Stats for the Outgoing Audio.
31+
- `getVideo()`: The list of Media Stats for the Outgoing Video.
32+
- `getScreenShare()`: The list of Media Stats for the Outgoing Screen Share.
33+
- `getIncomingStats()`: The list of Media Stats for Incoming media.
34+
- `getAudio()`: The list of Media Stats for the Incoming Audio.
35+
- `getVideo()`: The list of Media Stats for the Incoming Video.
36+
- `getScreenShare()`: The list of Media Stats for the Incoming Screen Share.
37+
- `getGeneratedAt()`: The date when the report was generated.
38+
- `getIncomingMediaStatsFromParticipant`: Gets the `IncomingMediaStats` for a `RemoteParticipant`.
39+
40+
Then, subscribe to the `addOnSampleReportedListener` event to get regular updates about the current media quality statistics:
41+
42+
```java
43+
mediaStatsCallFeature.addOnSampleReportedListener(handleSampleReportedListener);
44+
// Optional, set the interval for Media Stats report generation
45+
mediaStatsCallFeature.setSampleReportedIntervalInSeconds(15);
46+
47+
private void handleSampleReportedListener(MediaStatsReportEvent args) {
48+
// Obtain the Media Stats Report instance.
49+
MediaStatsReport report = args.getReport();
50+
51+
// Obtain the Outgoing Media Stats for Audio
52+
List<OutgoingAudioMediaStats> outgoingAudioMediaStats = report.getOutgoingMediaStats().getAudio();
53+
54+
// Obtain the Outgoing Media Stats for Video
55+
List<OutgoingVideoMediaStats> outgoingVideoMediaStats = report.getOutgoingMediaStats().getVideo();
56+
57+
// Obtain the Outgoing Media Stats for Screen Share
58+
List<OutgoingScreenShareMediaStats> outgoingScreenShareMediaStats = report.getOutgoingMediaStats().getScreenShare();
59+
60+
// Obtain the Incoming Media Stats for Audio
61+
List<IncomingAudioMediaStats> incomingAudioMediaStats = report.getIncomingMediaStats().getAudio();
62+
63+
// Obtain the Incoming Media Stats for Video
64+
List<IncomingVideoMediaStats> incomingVideoMediaStats = report.getIncomingMediaStats().getVideo();
65+
66+
// Obtain the Incoming Media Stats for Screen Share
67+
List<IncomingScreenShareMediaStats> incomingScreenShareMediaStats = report.getIncomingMediaStats().getScreenShare();
68+
}
69+
```
70+
71+
Also, `MediaStatsReport` have a helper method to obtain the `IncomingMediaStats` for a particular `RemoteParticipant`.
72+
For example, in order to get the `IncomingMediaStats` for all the `RemoteParticipants` in the call you can:
73+
74+
```java
75+
private void handleSampleReportedListener(MediaStatsReportEvent args) {
76+
List<RemoteParticipant> remoteParticipants = call.getRemoteParticipants();
77+
for (RemoteParticipant remoteParticipant : remoteParticipants) {
78+
{
79+
IncomingMediaStatsInfo incomingMediaStatsInfo = report.getIncomingMediaStatsFromParticipant(remoteParticipant.getIdentifier());
80+
// Obtain the Incoming Media Stats for Audio
81+
List<IncomingAudioMediaStats> incomingAudioMediaStats = incomingMediaStatsInfo.getAudio();
82+
83+
// Obtain the Incoming Media Stats for Video
84+
List<IncomingVideoMediaStats> incomingVideoMediaStats = incomingMediaStatsInfo.getVideo();
85+
86+
// Obtain the Incoming Media Stats for Screen Share
87+
List<IncomingScreenShareMediaStats> incomingScreenShareMediaStats = incomingMediaStatsInfo.getScreenShare();
88+
}
89+
}
90+
```
91+
92+
[!INCLUDE [native matrics](media-stats-native-metrics.md)]

‎articles/communication-services/concepts/voice-video-calling/includes/media-stats/media-stats-ios.md

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
---
2+
title: Azure Communication Services Media Stats (iOS)
3+
titleSuffix: An Azure Communication Services concept document
4+
description: Provides usage samples of the Media Stats feature iOS Native.
5+
author: jsaurezle-msft
6+
ms.author: jsaurezlee
7+
8+
services: azure-communication-services
9+
ms.date: 08/09/2023
10+
ms.topic: include
11+
ms.service: azure-communication-services
12+
ms.subservice: calling
13+
---
14+
15+
## Media quality statistics for ongoing call
16+
17+
[!INCLUDE [public-preview-notes](../../../../includes/public-preview-include.md)]
18+
19+
Media quality statistics is an extended feature of the core `Call` API. You first need to obtain the MediaStats feature API object:
20+
21+
```swift
22+
var mediaStatsCallFeature = self.call?.feature(Features.mediaStats)
23+
```
24+
25+
The Media Stats feature object have the following API structure:
26+
- `didReceiveSample`: Delegate method for listening for periodic reports of the Media Stats.
27+
- `sampleIntervalInSeconds`: Gets and sets the interval in seconds of the Media Stats report generation. If not specified, sdk use defaults.
28+
- A `MediaStatsReport` object that contains the definition of the Outgoing and Incoming Media Stats categorized by Audio, Video and Screen Share.
29+
- `OutgoingMediaStats`: The list of Media Stats for Outgoing media.
30+
- `audio`: The list of Media Stats for the Outgoing Audio.
31+
- `video`: The list of Media Stats for the Outgoing Video.
32+
- `screenShare`: The list of Media Stats for the Outgoing Screen Share.
33+
- `OncomingStats`: The list of Media Stats for Incoming media.
34+
- `audio`: The list of Media Stats for the Incoming Audio.
35+
- `video`: The list of Media Stats for the Incoming Video.
36+
- `screenShare`: The list of Media Stats for the Incoming Screen Share.
37+
- `generatedAt`: The date when the report was generated.
38+
- `incomingMediaStats`: Gets the `IncomingMediaStats` for a `RemoteParticipant`.
39+
40+
Then, subscribe to the `SampleReported` event to get regular updates about the current media quality statistics:
41+
42+
```swift
43+
// Optional, set the interval for Media Stats report generation
44+
mediaStatsCallFeature.sampleIntervalInSeconds = 15
45+
mediaStatsCallFeature.delegate = MediaStatsDelegate()
46+
47+
48+
public class MediaStatsDelegate : MediaStatsCallFeatureDelegate
49+
{
50+
public func mediaStatsCallFeature(_ mediaStatsCallFeature: MediaStatsCallFeature, didReceiveSample args: MediaStatsReportEventArgs) {
51+
let report = args.report
52+
53+
// Obtain the Outgoing Media Stats for Audio
54+
let outgoingAudioMediaStats = report.outgoingMediaStats.audio
55+
56+
// Obtain the Outgoing Media Stats for Video
57+
let outgoingVideoMediaStats = report.outgoingMediaStats.video
58+
59+
// Obtain the Outgoing Media Stats for Screen Share
60+
let outgoingScreenShareMediaStats = report.outgoingMediaStats.screenShare
61+
62+
// Obtain the Incoming Media Stats for Audio
63+
let incomingAudioMediaStats = report.incomingMediaStats.audio
64+
65+
// Obtain the Incoming Media Stats for Video
66+
let incomingVideoMediaStats = report.incomingMediaStats.video
67+
68+
// Obtain the Incoming Media Stats for Screen Share
69+
let incomingScreenShareMediaStats = report.incomingMediaStats.screenShare
70+
}
71+
}
72+
```
73+
74+
Also, `MediaStatsReport` have a helper method to obtain the `IncomingMediaStats` for a particular `RemoteParticipant`.
75+
For example, in order to get the `IncomingMediaStats` for all the `RemoteParticipants` in the call you can:
76+
77+
```swift
78+
public class MediaStatsDelegate : MediaStatsCallFeatureDelegate
79+
{
80+
public func mediaStatsCallFeature(_ mediaStatsCallFeature: MediaStatsCallFeature, didReceiveSample args: MediaStatsReportEventArgs) {
81+
let report = args.report
82+
83+
call.remoteParticipants.forEach{ remoteParticipant in
84+
let remoteIncomingMediaStats = report.incomingMediaStats(fromParticipant: remoteParticipant.identifier)
85+
86+
// Obtain the Incoming Media Stats for Audio
87+
let incomingAudioMediaStats = remoteIncomingMediaStats.audio
88+
89+
// Obtain the Incoming Media Stats for Video
90+
let incomingVideoMediaStats = remoteIncomingMediaStats.video
91+
92+
// Obtain the Incoming Media Stats for Screen Share
93+
let incomingScreenShareMediaStats = remoteIncomingMediaStats.screenShare
94+
}
95+
}
96+
}
97+
```
98+
99+
[!INCLUDE [native matrics](media-stats-native-metrics.md)]

‎articles/communication-services/concepts/voice-video-calling/includes/media-stats/media-stats-native-metrics.md

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
---
2+
title: Azure Communication Services Media Stats (Native)
3+
titleSuffix: An Azure Communication Services concept document
4+
description: Provides usage samples of the Media Stats feature for Native.
5+
author: jsaurezle-msft
6+
ms.author: jsaurezlee
7+
8+
services: azure-communication-services
9+
ms.date: 08/09/2023
10+
ms.topic: include
11+
ms.service: azure-communication-services
12+
ms.subservice: calling
13+
---
14+
15+
## Best practices
16+
If you want to collect this data for off-line inspection (after a call ends), it is recommended to collect this data and send it to your pipeline ingest after your call has ended. If you transmit this data during a call, it could use internet bandwidth that is needed to continue an Azure Communication Services call (especially when available bandwidth is low).
17+
18+
### Outgoing Audio metrics
19+
| Metric Name | Description | Comments |
20+
| ----------- | ----------- | -------- |
21+
| CodecName | codec name | |
22+
| Bitrate | audio send bitrate (bps) | General values are in the 24 kbps range (36-128 kbps typical) |
23+
| JitterInMs | packet jitter (milliseconds) | Lower is better. |
24+
| PacketsPerSecond | packet rate (packets/sec) | |
25+
| PacketsLostPerSecond | packet loss rate (packets/sec) | Lower is better. |
26+
| RoundTripTimeInMs | round-trip time (milliseconds) | Lower is better. It's calculated from RTCP Receiver Report. A round trip time of 200 ms or less is recommended. |
27+
| PairRoundTripTimeInMs | round-trip time (milliseconds) | Lower is better. It's similar to rttInMS but is calculated from STUN connectivity check. A round trip time of 200 ms or less is recommended. |
28+
| AvailableBitrate | bandwidth estimation (bps) | |
29+
| AudioInputLevel | audio volume level from microphone | The value ranges from 0-65536. 0 represents silence |
30+
31+
### Incoming Audio metrics
32+
| Metric Name | Description | Comments |
33+
| ----------- | ----------- | -------- |
34+
| CodecName | codec name | |
35+
| Bitrate | audio receive bitrate (bps) | General values are in the 24 kbps range (36-128 kbps typical) |
36+
| JitterInMs | packet jitter (milliseconds) | Lower is better. |
37+
| PacketsPerSecond | packet rate (packets/sec) | |
38+
| PacketsLostPerSecond | packet loss rate (packets/sec) | Lower is better. |
39+
| RoundTripTimeInMs | round-trip time (milliseconds) | Lower is better. It's calculated from RTCP Receiver Report. A round trip time of 200 ms or less is recommended. |
40+
| PairRoundTripTimeInMs | round-trip time (milliseconds) | Lower is better. It is calculated from STUN connectivity check. A round trip time of 200 ms or less is recommended. |
41+
| AvailableBitrate | bandwidth estimation (bps) | |
42+
| JitterBufferInMs | jitter buffer (milliseconds) | Lower is better. The jitter buffer is used for smooth playout. This value is how long the packets of the samples stay in the jitter buffer. |
43+
| AudioOutputLevel | audio volume level from receiving stream | The value ranges from 0-65536. 0 represents silence. |
44+
| HealedRatio | ratio of concealedSamples(except silentConcealedSamples) to total received samples | Information only. |
45+
46+
### Outgoing Video metrics
47+
| Metric Name | Description | Comments |
48+
| ----------- | ----------- | -------- |
49+
| CodecName | codec name | |
50+
| Bitrate | video send bitrate (bps) | |
51+
| JitterInMs | packet jitter (milliseconds) | Lower is better. |
52+
| PacketsPerSecond | packet rate (packets/sec) | |
53+
| PacketsLostPerSecond | packet loss rate (packets/sec) | Lower is better. |
54+
| RoundTripTimeInMs | round-trip time (milliseconds) | Lower is better. It is calculated from RTCP Receiver Report. A round trip time of 200 ms or less is recommended. |
55+
| PairRoundTripTimeInMs | round-trip time (milliseconds) | Lower is better. It is similar to rttInMS but is calculated from STUN connectivity check. A round trip time of 200 ms or less is recommended. |
56+
| AvailableBitrate | bandwidth estimation (bps) | 1.5 Mbps or higher is recommended for high-quality video for upload/download. |
57+
| FrameRateInput | frame rate originating from the video source (frames/sec) | |
58+
| FrameWidthInput | frame width of the last frame originating from video source (pixel) | |
59+
| FrameHeightInput | frame height of the last frame originating from video source (pixel) | |
60+
| FrameRateEncoded | frame rate successfully encoded for the RTP stream (frames/sec) | |
61+
| FrameRateSent | frame rate sent on the RTP stream (frames/sec) | |
62+
| FrameWidthSent | frame width of the encoded frame (pixel) | |
63+
| FrameHeightSent | frame height of the encoded frame (pixel) | |
64+
| FramesSent | frames sent on the RTP stream | |
65+
| FramesEncoded | frames successfully encoded for the RTP stream | |
66+
| KeyFramesEncoded | key frames successfully encoded for the RTP stream | |
67+
68+
### Incoming Video metrics
69+
| Metric Name | Description | Comments |
70+
| ----------- | ----------- | -------- |
71+
| CodecName | codec name | |
72+
| Bitrate | video receive bitrate (bps) | |
73+
| JitterInMs | packet jitter (milliseconds) | Lower is better. |
74+
| PacketsPerSecond | packet rate (packets/sec) | |
75+
| PacketsLostPerSecond | packet loss rate (packets/sec) | Lower is better. |
76+
| RoundTripTimeInMs | round-trip time (milliseconds) | Lower is better. It is calculated from RTCP Receiver Report. A round trip time of 200 ms or less is recommended. |
77+
| PairRoundTripTimeInMs | round-trip time (milliseconds) | Lower is better. A round trip time of 200 ms or less is recommended. |
78+
| AvailableBitrate | bandwidth estimation (bps) | 1.5 Mbps or higher is recommended for high-quality video for upload/download. |
79+
| JitterBufferInMs | jitter buffer (milliseconds) | Lower is better. The jitter buffer is used for smooth playout. This value is how long the packets of the frame stay in the jitter buffer. |
80+
| StreamId | stream id | The streamId value corresponds to id in VideoStreamCommon. It can be used to match the sender. |
81+
| FrameRateOutput | frame rate output (frames/sec) | |
82+
| FrameRateDecoded | frame rate correctly decoded for the RTP stream (frames/sec) | |
83+
| FrameRateReceived | frame rate received on the RTP stream (frames/sec) | |
84+
| FrameWidthReceived | frame width of the decoded frame (pixel) | |
85+
| FrameHeightReceived | frame height of the decoded frame (pixel) | |
86+
| LongestFreezeDurationInMs | longest freeze duration (milliseconds) | |
87+
| TotalFreezeDurationInMs | total freeze duration (milliseconds) | |
88+
| FramesReceived | total number of frames received on the RTP stream | |
89+
| FramesDecoded | total number of frames correctly decoded for the RTP stream | |
90+
| FramesDropped | total number of frames dropped | |
91+
| KeyFramesDecoded | total number of key frames correctly decoded for the RTP stream | |
92+
93+
### ScreenShare Send metrics
94+
Currently stats fields are the same as *Video Send metrics*
95+
96+
### ScreenShare Receive metrics
97+
Currently stats fields are the same as *Video Receive metrics*

0 commit comments

Comments
 (0)