Skip to content

Commit f66fcf0

Browse files
JamesJBarnettJamesJBarnett
authored
Merge pull request #291376 from ahammer/adamhammer/rtt_updates
Real Time Text Draft PR
2 parents bb6cd74 + e45e9cf commit f66fcf0

File tree

5 files changed

+344
-0
lines changed

5 files changed

+344
-0
lines changed

‎articles/communication-services/concepts/voice-video-calling/real-time-text.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
---
2+
title: Real Time Text (RTT) Overview
3+
titleSuffix: An Azure Communication Services concept document
4+
description: Learn about Real Time Text (RTT) in Azure Communication Services.
5+
author: ahammer
6+
ms.date: 12/4/2024
7+
ms.author: adamhammer
8+
ms.topic: conceptual
9+
ms.service: azure-communication-services
10+
ms.subservice: calling
11+
---
12+
13+
14+
15+
16+
# Real Time Text (RTT) Overview
17+
18+
>[!NOTE]
19+
>RTT is an accessibility compliance requirement for voice and video platforms in the EU. You can find more information about this here: [Directive 2019/882](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32019L0882)
20+
21+
Real Time Text (RTT) provides developers with the ability to transmit text in near real-time during a call. This feature is designed to empower individuals who have difficulty speaking, ensuring their text messages are displayed prominently to other meeting participants, similar to spoken communication. RTT enhances accessibility by allowing participants to communicate effectively through typed messages that are broadcast instantly character by character, without having to press a "send" key.
22+
23+
## Common Use Cases
24+
25+
### Building Accessible Experiences
26+
27+
RTT aids in creating inclusive communication platforms by enabling users with speech impairments to participate fully in calls. By providing a text-based communication channel that operates in real-time, RTT ensures that all participants can engage equally, regardless of their ability to speak.
28+
29+
### Enhancing Communication Clarity
30+
31+
In scenarios where speech clarity is compromised due to background noise or technical limitations, RTT serves as a reliable alternative to convey messages clearly. This ensures that critical information is communicated without misunderstandings.
32+
33+
## When to Use Real Time Text (RTT)
34+
35+
- **Accessibility Requirements:** When developing applications that need to comply with accessibility standards, enabling RTT ensures that users with speech difficulties can participate effectively.
36+
- **High-Noise Environments:** In settings where audio quality may be affected by background noise, RTT provides a clear and reliable means of communication.
37+
38+
## RealTimeTextInfo/Details Class
39+
40+
The `RealTimeTextInfo` (or `RealTimeTextDetails` on certain platforms) class is pivotal in managing RTT messages. It encapsulates all necessary information about each RTT message, including the sender, content, sequence identifier, result type, timestamps, and whether the message originates locally.
41+
42+
### Properties and Fields
43+
44+
- **Sender:** Provides information about the user who sent the RTT message.
45+
- **SequenceId:** A unique identifier that maintains the order of messages.
46+
- **Text:** The actual content of the RTT message.
47+
- **ResultType/Kind:** Indicates whether the message is partial (`ResultType`) or finalized (`Kind`), determining if it can be edited.
48+
- **ReceivedTime:** Timestamp marking when the message was received.
49+
- **UpdatedTime:** Timestamp indicating the last update to the message.
50+
- **IsLocal:** A boolean flag indicating if the message was sent by the local user.
51+
52+
### Usage
53+
54+
Developers can subscribe to RTT events through a single event subscription, allowing them to manage their own list and ordering of RTT messages. This approach aligns with the Captions API, facilitating easier integration and maintenance. The UI is responsible for binding the data to the screen, managing the display list, and handling text input interactions based on message finalization.
55+
56+
### Privacy Concerns
57+
Real-Time Text (RTT) is only available during the call or meeting. Azure Communication Services doesn't store these text exchanges anywhere. Many countries/regions and states have laws and regulations that apply to the storing of data. It is your responsibility to use RTT in compliance with the law should you choose to store any of the data generated through RTT. You must obtain consent from the parties involved in a manner that complies with the laws applicable to each participant.
58+
59+
Interoperability between Azure Communication Services and Microsoft Teams enables your applications and users to participate in Teams calls, meetings, and chats. It is your responsibility to ensure that the users of your application are notified when RTT is enabled in a Teams call or meeting and being stored.
60+
61+
Microsoft indicates to you via the Azure Communication Services API that recording or RTT has commenced, and you must communicate this fact, in real-time, to your users within your application's user interface. You agree to indemnify Microsoft for all costs and damages incurred due to your failure to comply with this obligation.
62+
63+
## Next Steps
64+
65+
- Get started with a [Quickstart Guide](../../quickstarts/voice-video-calling/get-started-with-real-time-text.md)
66+
- Learn more about [Closed captions](../../concepts/voice-video-calling/closed-captions.md)
67+
- Explore the [UI Library](../ui-library/ui-library-overview.md)

‎articles/communication-services/quickstarts/voice-video-calling/get-started-with-real-time-text.md

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
---
2+
title: Real Time Text
3+
titleSuffix: An Azure Communication Services how-to document
4+
description: Provides a how-to guide for the Real Time Text feature, enabling near-real-time text communication during calls.
5+
author: ahammer
6+
ms.author: adamhammer
7+
ms.service: azure-communication-services
8+
ms.subservice: calling
9+
ms.topic: how-to
10+
ms.date: 12/4/2024
11+
ms.custom: template-how-to
12+
zone_pivot_groups: acs-programming-languages-java-swift-csharp
13+
---
14+
15+
16+
17+
# Real Time Text
18+
19+
[!INCLUDE [Public Preview Disclaimer](../../includes/public-preview-include-document.md)]
20+
21+
Learn how to integrate Real Time Text (RTT) into your calling applications to enhance accessibility and ensure that all participants can communicate effectively during meetings.
22+
23+
RTT allows users who have difficulty speaking to participate actively by typing their messages, which are then broadcast in near real-time to other meeting participants. This feature operates seamlessly alongside existing captions and ensures that typed messages are promptly delivered without disrupting the flow of conversation.
24+
25+
## Real Time Text Feature Overview
26+
27+
Real Time Text (RTT) is designed to facilitate communication for users who may have difficulty speaking during calls. By allowing users to type their messages, RTT ensures that everyone in the meeting can stay engaged and informed. Messages are transmitted over Data Channels (ID 24) and are always active, appearing automatically when the first message is sent.
28+
29+
On supported platforms, RTT data can be displayed alongside captions derived from Speech to Text, providing a comprehensive view of all communications during a call.
30+
31+
## Naming Conventions
32+
33+
Different platforms may use varying terminology for RTT-related properties. Below is a summary of these differences:
34+
35+
| Mobile (Android/iOS) | Windows (C#) |
36+
| -------------------- | ------------- |
37+
| **Type** | **Kind** |
38+
| **Info** | **Details** |
39+
40+
These aliases are functionally equivalent and are used to maintain consistency across different platforms.
41+
42+
## RealTimeTextInfo/Details Class
43+
44+
The `RealTimeTextInfo` (or `RealTimeTextDetails` on Windows) class encapsulates information about each RTT message. Below are the key properties:
45+
46+
| Property | Description |
47+
| ----------------- | ----------------------------------------------------- |
48+
| `SequenceId` | Unique identifier for the message sequence. |
49+
| `Text` | The content of the RTT message. |
50+
| `Sender` | Information about the sender of the message. |
51+
| `ResultType`/<br>`Kind` | Indicates whether the message is partial or final. |
52+
| `IsLocal` | Determines if the message was sent by the local user. |
53+
| `ReceivedTime` | Timestamp when the message was received. |
54+
| `UpdatedTime` | Timestamp when the message was last updated. |
55+
56+
57+
::: zone pivot="programming-language-java"
58+
[!INCLUDE [Real Time Text with Android](./includes/real-time-text/real-time-text-android.md)]
59+
::: zone-end
60+
61+
::: zone pivot="programming-language-swift"
62+
[!INCLUDE [Real Time Text with iOS](./includes/real-time-text/real-time-text-ios.md)]
63+
::: zone-end
64+
65+
::: zone pivot="programming-language-csharp"
66+
[!INCLUDE [Real Time Text with Windows](./includes/real-time-text/real-time-text-windows.md)]
67+
::: zone-end
68+
69+
70+
## Next steps
71+
72+
For more information, see the following articles:
73+
74+
- [Azure Communication Services Calling Documentation](../../concepts/voice-video-calling/calling-sdk-features.md)

‎articles/communication-services/quickstarts/voice-video-calling/includes/real-time-text/real-time-text-android.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
---
2+
author: ahammer
3+
ms.service: azure-communication-services
4+
ms.topic: include
5+
ms.subservice: calling
6+
ms.date: 12/4/2024
7+
ms.author: adam.hammer
8+
---
9+
10+
## Models
11+
12+
| Name | Description |
13+
| ------------------ | ------------------------------------------------ |
14+
| `RealTimeTextInfo` | Represents a real-time text message entry, including sender information, message content, sequence ID, and status. |
15+
16+
## Get Real Time Text Feature
17+
18+
To access the Real Time Text feature, retrieve it from the `Call` object:
19+
20+
```java
21+
RealTimeTextCallFeature rttFeature = call.feature(Features.REAL_TIME_TEXT);
22+
```
23+
24+
## Feature Usage
25+
26+
### Sending Real Time Text Messages
27+
28+
Bind a text input field to the `send()` method to transmit messages as the user types:
29+
30+
```java
31+
EditText messageEditText = findViewById(R.id.messageEditText);
32+
messageEditText.addTextChangedListener(new TextWatcher() {
33+
@Override
34+
public void afterTextChanged(Editable s) {
35+
String text = s.toString();
36+
rttFeature.send(text);
37+
}
38+
// Other overridden methods...
39+
});
40+
```
41+
42+
### Receiving Real Time Text Messages
43+
44+
Subscribe to the `OnInfoReceived` event to handle incoming messages:
45+
46+
```java
47+
rttFeature.addOnInfoReceivedListener((eventArgs) -> {
48+
RealTimeTextInfo info = eventArgs.getInfo();
49+
50+
// Update your message list with the new info
51+
updateMessageList(info);
52+
53+
// Clear the text input if the message is local and finalized
54+
if (info.isLocal() && info.getResultType() == RealTimeTextResultType.FINAL) {
55+
messageEditText.getText().clear();
56+
}
57+
});
58+
```
59+
60+
## RealTimeTextInfo Class
61+
62+
The `RealTimeTextInfo` class provides detailed information about each real-time text message:
63+
64+
- **Sender**: Information about who sent the message.
65+
- **SequenceId**: Unique identifier for the message.
66+
- **Text**: The content of the message.
67+
- **ResultType**: Indicates if the message is partial or finalized.
68+
- **ReceivedTime**: Timestamp when the message was received.
69+
- **UpdatedTime**: Timestamp when the message was last updated.
70+
- **IsLocal**: Indicates if the message was sent by the local user.

‎articles/communication-services/quickstarts/voice-video-calling/includes/real-time-text/real-time-text-ios.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
---
2+
author: ahammer
3+
ms.service: azure-communication-services
4+
ms.topic: include
5+
ms.subservice: calling
6+
ms.date: 12/4/2024
7+
ms.author: adam.hammer
8+
---
9+
10+
## Models
11+
12+
| Name | Description |
13+
| ------------------ | ------------------------------------------------ |
14+
| `RealTimeTextInfo` | Represents a real-time text message entry, including sender information, message content, sequence ID, and status. |
15+
16+
## Get Real Time Text Feature
17+
18+
Access the Real Time Text feature from your `Call` object:
19+
20+
```swift
21+
let rttFeature = call.feature(Features.realTimeText)
22+
```
23+
24+
## Feature Usage
25+
26+
### Sending Real Time Text Messages
27+
28+
Bind a text input field to the `send` method to transmit messages as the user types:
29+
30+
```swift
31+
@State var messageText: String = ""
32+
33+
TextField("Type your message", text: $messageText)
34+
.onChange(of: messageText) { newText in
35+
rttFeature?.send(newText)
36+
}
37+
```
38+
39+
### Receiving Real Time Text Messages
40+
41+
Subscribe to the `OnInfoReceived` event to handle incoming messages:
42+
43+
```swift
44+
rttFeature?.addOnInfoReceivedListener { eventArgs in
45+
if let info = eventArgs.info {
46+
// Update your message list with the new info
47+
updateMessageList(info)
48+
49+
// Clear the text input if the message is local and finalized
50+
if info.isLocal && info.resultType == .final {
51+
self.messageText = ""
52+
}
53+
}
54+
}
55+
```
56+
57+
## RealTimeTextInfo Class
58+
59+
The `RealTimeTextInfo` class provides detailed information about each real-time text message:
60+
61+
- **Sender**: Information about who sent the message.
62+
- **SequenceId**: Unique identifier for the message.
63+
- **Text**: The content of the message.
64+
- **ResultType**: Indicates if the message is partial or finalized.
65+
- **ReceivedTime**: Timestamp when the message was received.
66+
- **UpdatedTime**: Timestamp when the message was last updated.
67+
- **IsLocal**: Indicates if the message was sent by the local user.

‎articles/communication-services/quickstarts/voice-video-calling/includes/real-time-text/real-time-text-windows.md

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
---
2+
author: ahammer
3+
ms.service: azure-communication-services
4+
ms.topic: include
5+
ms.subservice: calling
6+
ms.date: 12/4/2024
7+
ms.author: adam.hammer
8+
---
9+
10+
## Models
11+
12+
| Name | Description |
13+
| ------------------ | ------------------------------------------------ |
14+
| `RealTimeTextDetails` | Represents a real-time text message entry, including sender information, message content, sequence ID, and status. |
15+
16+
## Get Real Time Text Feature
17+
18+
Retrieve the Real Time Text feature from the `Call` object:
19+
20+
```csharp
21+
RealTimeTextCallFeature rttFeature = call.GetRealTimeTextCallFeature();
22+
```
23+
24+
## Feature Usage
25+
26+
### Sending Real Time Text Messages
27+
28+
Connect a text input field to the `Send` method to transmit messages as the user types:
29+
30+
```csharp
31+
TextBox messageTextBox = new TextBox();
32+
messageTextBox.TextChanged += (sender, args) => {
33+
string text = messageTextBox.Text;
34+
rttFeature.Send(text);
35+
};
36+
```
37+
38+
### Receiving Real Time Text Messages
39+
40+
Subscribe to the `DetailsReceived` event to handle incoming messages:
41+
42+
```csharp
43+
rttFeature.DetailsReceived += (sender, e) => {
44+
RealTimeTextDetails details = e.Details;
45+
46+
// Update your message list with the new details
47+
UpdateMessageList(details);
48+
49+
// Clear the text input if the message is local and finalized
50+
if (details.IsLocal && details.Kind == RealTimeTextResultKind.Final) {
51+
messageTextBox.Text = string.Empty;
52+
}
53+
};
54+
```
55+
56+
## RealTimeTextDetails Class
57+
58+
The `RealTimeTextDetails` class provides comprehensive information about each real-time text message:
59+
60+
- **Sender**: Information about who sent the message.
61+
- **SequenceId**: Unique identifier for the message.
62+
- **Text**: The content of the message.
63+
- **Kind**: Indicates if the message is partial or finalized.
64+
- **ReceivedTime**: Timestamp when the message was received.
65+
- **UpdatedTime**: Timestamp when the message was last updated.
66+
- **IsLocal**: Indicates if the message was sent by the local user.

0 commit comments

Comments
 (0)