|
| 1 | +--- |
| 2 | +title: AI in Azure Communication Services |
| 3 | +titleSuffix: An Azure Communication Services concept document |
| 4 | +description: Learn about Communication Services AI concepts |
| 5 | +author: chpalm |
| 6 | +manager: sundraman |
| 7 | +services: azure-communication-services |
| 8 | +ms.author: chpalm |
| 9 | +ms.date: 07/10/2024 |
| 10 | +ms.topic: conceptual |
| 11 | +ms.service: azure-communication-services |
| 12 | +--- |
| 13 | + |
| 14 | +# Artificial intelligence (AI) overview |
| 15 | + |
| 16 | +Artificial intelligence (AI) technologies can be useful for a wide variety of communication experiences. This concept page summarizes availability of AI and AI-adjacent features in Azure Communication Services. AI features can be split into three categories: |
| 17 | + |
| 18 | +- **Accessors.** APIs that allow you to access Azure Communication data for the purposes of integrating your own separate transformations and bots. |
| 19 | +- **Transformers.** APIs that provide a built-in transformation of communication data using a machine learning or language model. |
| 20 | +- **Bots.** APIs that implement bots that directly communicate with end-users, typically blending structured programming with language models. |
| 21 | + |
| 22 | +Typical communication scenarios integrating these capabilities: |
| 23 | + |
| 24 | +- Transforming audio speech content into text transcriptions |
| 25 | +- Transforming a video feed to blur the user's background |
| 26 | +- Operating a chat or voice bot that responds to human conversation |
| 27 | +- Transforming a corpus of text chat and meeting transcriptions into summaries. This experience might involve a generative AI interface in which a user asks, "summarize all conversations between me and user Joe." |
| 28 | + |
| 29 | +## Messaging: SMS, Chat, Email, WhatsApp |
| 30 | + |
| 31 | +Azure Communication Services capabilities for asynchronous messaging share common patterns for integrating AI listed here. |
| 32 | + |
| 33 | +| Feature | Accessor | Transformer | Bot | Description | |
| 34 | +|--|--|--|--|--| |
| 35 | +| REST APIs and SDKs| ✅ | | | The messaging services center around REST APIs and server-oriented SDKs. You can use these SDKs to export content to an external datastore and attach a language model to summarize conversations. Or you can use the SDKs to integrate a bot that directly engages with human users. | |
| 36 | +| WhatsApp Message Analysis | | ✅ | | The Azure Communication Service messaging APIs for WhatsApp provide a built-in integration with Azure OpenAI that analyses and annotates messages. This integration can detect the user’s language, recognize their intent, and extract key phrases. | |
| 37 | +| [Azure Bot – Chat Channel Integration](../quickstarts/chat/quickstart-botframework-integration.md) | | | ✅ | The Azure Communication Service chat system is directly integrated with Azure Bot services. This integration simplifies creating chat bots that engage with human users.| |
| 38 | + |
| 39 | +## Voice, Video, and Telephony |
| 40 | + |
| 41 | +The patterns for integrating AI into the voice and video system are summarized here. |
| 42 | + |
| 43 | +| Feature | Accessor | Transformer | Bot | Description | |
| 44 | +|--|--|--|--|--| |
| 45 | +| [Call Automation REST APIs and SDKs](../concepts/call-automation/call-automation.md) | ✅ | ✅ | | Call Automation APIs include both accessors and transformers, with REST APIs for playing audio files and recognizing a user’s response. The `recognize` APIs integrate Azure Bot Services to transform users’ audio content into text for easier processing by your service. The most common scenario for these APIs is implementing voice bots, sometimes called interactive voice response (IVR). | |
| 46 | +| [Microsoft Copilot Studio](https://learn.microsoft.com/microsoft-copilot-studio/voice-overview) | | ✅ | ✅ | Copilot studio is directly integrated with Azure Communication Services telephony. This integration is designed for voice bots and IVR. | |
| 47 | +| [Azure Portal Copilot](https://learn.microsoft.com/microsoft-copilot-studio/voice-overview) | | ✅ | ✅ | Copilot studio is directly integrated with Azure Communication Services telephony. This integration is designed for voice bots and IVR. | |
| 48 | +| [Client Raw Audio and Video](../concepts/voice-video-calling/media-access.md) | ✅ | | | The Calling client SDK provides APIs for accessing and modifying the raw audio and video feed. An example scenario is taking the video feed, detecting the human speaker and their background, and customizing that background. | |
| 49 | +| [Client Background effects](../quickstarts/voice-video-calling/get-started-video-effects.md?pivots=platform-web)| | ✅ | | The Calling client SDKs provides APIs for blurring or replacing a user’s background. | |
| 50 | +| [Client Captions](../concepts/voice-video-calling/closed-captions.md) | | ✅ | | The Calling client SDK provides APIs for real-time closed captions. These internally integrate Azure Cognitive Services to transform audio content from the call into text in real-time. | |
| 51 | +| [Client Noise Enhancement and Effects](../tutorials/audio-quality-enhancements/add-noise-supression.md?pivots=platform-web) | | ✅ | | The Calling client SDK integrates a [DeepVQE](https://arxiv.org/abs/2306.03177) machine learning model to improve audio quality through echo cancellation, noise suppression, and dereverberation. This transformation is toggled on and off using the client SDK. | |
0 commit comments