Skip to content

Commit 1a0c936

Browse files
JuliakoJuliako
committed
emotions
1 parent 284b745 commit 1a0c936

File tree

2 files changed

+121
-0
lines changed

2 files changed

+121
-0
lines changed

‎articles/azure-video-indexer/emotions-detection.md

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
---
2+
title: Azure Video Indexer emotions detection overview
3+
titleSuffix: Azure Video Indexer
4+
description: This article gives an overview of an Azure Video Indexer emotions detection.
5+
author: juliako
6+
ms.author: juliako
7+
manager: femila
8+
ms.service: azure-video-indexer
9+
ms.date: 06/15/2022
10+
ms.topic: article
11+
---
12+
13+
# Emotions detection
14+
15+
Emotion detection is an Azure Video Indexer AI feature that automatically detects emotions a video's transcript lines. Each sentence can either be detected as "Anger", "Fear", "Joy", "Neutral", and "Sad". The model works on text only (labeling emotions in video transcripts.) This model doesn't infer the emotional state of people, may not perform where input is ambiguous or unclear, like sarcastic remarks. Thus, the model shouldn't be used for things like assessing employee performance or the emotional state of a person.
16+
17+
The model doesn't have context of the input data, which can impact its accuracy. To increase the accuracy, it's recommended for the input data to be in a clear and unambiguous format.
18+
19+
## Prerequisites
20+
21+
Review [Transparency Note overview](/legal/azure-video-indexer/transparency-note?context=/azure/azure-video-indexer/context/context)
22+
23+
## General principles
24+
25+
There are many things you need to consider when deciding how to use and implement an AI-powered feature:
26+
27+
- Will this feature perform well in my scenario? Before deploying emotions detection into your scenario, test how it performs using real-life data and make sure it can deliver the accuracy you need.
28+
- Are we equipped to identify and respond to errors? AI-powered products and features won't be 100% accurate, so consider how you'll identify and respond to any errors that may occur.
29+
30+
## View the insight
31+
32+
When working on the website the insights are displayed in the **Insights** tab. They can also be generated in a categorized list in a JSON file that includes the id, type, and a list of instances it appeared at, with their time and confidence.
33+
34+
To display the instances in a JSON file, do the following:
35+
36+
1. Select Download -> Insights (JSON).
37+
1. Copy the text and paste it into an online JSON viewer.
38+
39+
```json
40+
"emotions": [
41+
{
42+
"id": 1,
43+
"type": "Sad",
44+
"instances": [
45+
{
46+
"confidence": 0.5518,
47+
"adjustedStart": "0:00:00",
48+
"adjustedEnd": "0:00:05.75",
49+
"start": "0:00:00",
50+
"end": "0:00:05.75"
51+
},
52+
53+
```
54+
55+
To download the JSON file via the API, use the [Azure Video Indexer developer portal](https://api-portal.videoindexer.ai/).
56+
57+
> [!NOTE]
58+
> Emotions detection is language independent, however if the transcript is not in English, it is first being translated to English and only then the model is applied. This may cause a reduced accuracy in emotions detection for non English languages.
59+
60+
## Emotions detection components
61+
62+
During the emotions detection procedure, the transcript of the video is processed, as follows:
63+
64+
|Component |Definition |
65+
|---|---|
66+
|Source language |The user uploads the source file for indexing. |
67+
|Transcription API |The audio file is sent to Cognitive Services and the translated transcribed output is returned. If a language has been specified, it is processed. |
68+
|Emotions detection |Each sentence is sent to the emotions detection model. The model produces the confidence level of each emotion. If the confidence level exceeds a specific threshold, and there is no ambiguity between positive and negative emotions, the emotion is detected. In any other case, the sentence is labeled as neutral.|
69+
|Confidence level |The estimated confidence level of the detected emotions is calculated as a range of 0 to 1. The confidence score represents the certainty in the accuracy of the result. For example, an 82% certainty is represented as an 0.82 score. |
70+
71+
## Example use cases
72+
73+
* Personalization of keywords to match customer interests, for example websites about England posting promotions about English movies or festivals.
74+
* Deep-searching archives for insights on specific keywords to create feature stories about companies, personas or technologies, for example by a news agency.
75+
76+
## Considerations and limitations when choosing a use case
77+
78+
Below are some considerations to keep in mind when using emotions detection:
79+
80+
* When uploading a file always use high quality audio and video content.
81+
82+
When used responsibly and carefully emotions detection is a valuable tool for many industries. To respect the privacy and safety of others, and to comply with local and global regulations, we recommend the following:  
83+
84+
* Always respect an individual’s right to privacy, and only ingest media for lawful and justifiable purposes.  
85+
Don't purposely disclose inappropriate media showing young children or family members of celebrities or other content that may be detrimental or pose a threat to an individual’s personal freedom.  
86+
* Commit to respecting and promoting human rights in the design and deployment of your analyzed media.  
87+
* When using 3rd party materials, be aware of any existing copyrights or permissions required before distributing content derived from them. 
88+
* Always seek legal advice when using media from unknown sources. 
89+
* Always obtain appropriate legal and professional advice to ensure that your uploaded media is secured and have adequate controls to preserve the integrity of your content and to prevent unauthorized access.    
90+
* Provide a feedback channel that allows users and individuals to report issues with the service.  
91+
* Be aware of any applicable laws or regulations that exist in your area regarding processing, analyzing, and sharing media containing people. 
92+
* Keep a human in the loop. Don't use any solution as a replacement for human oversight and decision-making.  
93+
* Fully examine and review the potential of any AI model you're using to understand its capabilities and limitations. 
94+
95+
## Next steps
96+
97+
### Learn More about Responsible AI
98+
99+
- [Microsoft Responsible AI principles](https://www.microsoft.com/ai/responsible-ai?activetab=pivot1%3aprimaryr6)
100+
- [Microsoft Responsible AI resources](https://www.microsoft.com/ai/responsible-ai-resources)
101+
- [Microsoft Azure Learning courses on Responsible AI](/training/paths/responsible-ai-business-principles/)
102+
- [Microsoft Global Human Rights Statement](https://www.microsoft.com/corporate-responsibility/human-rights-statement?activetab=pivot_1:primaryr5)
103+
104+
### Contact us
105+
106+
`[email protected]`
107+
108+
## Azure Video Indexer insights
109+
110+
View some other Azure Video Insights:
111+
112+
- [Audio effects detection](audio-effects-detection.md)
113+
- [Face detection](face-detection.md)
114+
- [OCR](ocr.md)
115+
- [Keywords extraction](keywords.md)
116+
- [Transcription, Translation & Language identification](transcription-translation-lid.md)
117+
- [Named entities](named-entities.md)
118+
- [Observed people tracking & matched persons](observed-matched-people.md)
119+
- [Topics inference](topics-inference.md)

‎articles/azure-video-indexer/toc.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,8 @@
8080
href: observed-matched-people.md
8181
- name: Topics inference
8282
href: topics-inference.md
83+
- name: Emotions detection
84+
href: emotions-detection.md
8385
- name: Customizing content models
8486
displayName: customize, customizing
8587
items:

0 commit comments

Comments
 (0)