Skip to content

Commit c771605

Browse files
authored
Merge pull request #79822 from diberry/0617-qna-updates
[Cogsvcs] QnA Maker - threshold
2 parents de9c56e + 77a4b27 commit c771605

File tree

2 files changed

+45
-5
lines changed

2 files changed

+45
-5
lines changed

articles/cognitive-services/QnAMaker/Concepts/confidence-score.md

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,13 @@ title: Confidence Score - QnA Maker
33
titleSuffix: Azure Cognitive Services
44
description: The confidence score indicates the confidence that the answer is the right match for the given user query.
55
services: cognitive-services
6-
author: tulasim88
6+
author: diberry
77
manager: nitinme
88
ms.service: cognitive-services
99
ms.subservice: qna-maker
1010
ms.topic: article
11-
ms.date: 04/05/2019
12-
ms.author: tulasim
11+
ms.date: 06/17/2019
12+
ms.author: diberry
1313
ms.custom: seodec18
1414
---
1515
# Confidence score of a QnA Maker knowledge base
@@ -40,7 +40,7 @@ The following table indicates typical confidence associated for a given score.
4040
|0|No match, so the answer is not returned.|"How much does the service cost"|
4141

4242
## Choose a score threshold
43-
The table above shows the scores that are expected on most KBs. However, since every KB is different, and has different types of words, intents and goals- we recommend you test and choose the threshold that best works for you. By default the threshold is set to 0, so that all possible answers are returned. The recommended threshold that should work for most KBs, is **50**.
43+
The table above shows the scores that are expected on most KBs. However, since every KB is different, and has different types of words, intents, and goals- we recommend you test and choose the threshold that best works for you. By default the threshold is set to 0, so that all possible answers are returned. The recommended threshold that should work for most KBs, is **50**.
4444

4545
When choosing your threshold, keep in mind the balance between Accuracy and Coverage, and tweak your threshold based on your requirements.
4646

@@ -51,6 +51,12 @@ When choosing your threshold, keep in mind the balance between Accuracy and Cove
5151
> [!NOTE]
5252
> Newer versions of QnA Maker include improvements to scoring logic, and could affect your threshold. Any time you update the service, make sure to test and tweak the threshold if necessary. You can check your QnA Service version [here](https://www.qnamaker.ai/UserSettings), and see how to get the latest updates [here](../How-To/troubleshooting-runtime.md).
5353
54+
## Set threshold
55+
56+
Set the threshold score as a property of the [GenerateAnswer API JSON body](../how-to/metadata-generateanswer-usage.md#generateanswer-request-configuration). This means you set it for each call to GenerateAnswer.
57+
58+
From the bot framework, set the score as part of the options object with [C#](../how-to/metadata-generateanswer-usage.md?#use-qna-maker-with-a-bot-in-c) or [Node.js](../how-to/metadata-generateanswer-usage.md?#use-qna-maker-with-a-bot-in-nodejs).
59+
5460
## Improve confidence scores
5561
To improve the confidence score of a particular response to a user query, you can add the user query to the knowledge base as an alternate question on that response. You can also use case-insensitive [word alterations](https://docs.microsoft.com/rest/api/cognitiveservices/qnamaker/alterations/replace) to add synonyms to keywords in your KB.
5662

articles/cognitive-services/QnAMaker/How-To/metadata-generateanswer-usage.md

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ manager: nitinme
88
ms.service: cognitive-services
99
ms.subservice: qna-maker
1010
ms.topic: article
11-
ms.date: 05/30/2019
11+
ms.date: 06/17/2019
1212
ms.author: tulasim
1313
---
1414

@@ -140,6 +140,40 @@ A successful response returns a status of 200 and a JSON response.
140140
}
141141
```
142142

143+
## Use QnA Maker with a bot in C#
144+
145+
The bot framework provides access to the QnA Maker's properties:
146+
147+
```csharp
148+
using Microsoft.Bot.Builder.AI.QnA;
149+
var metadata = new Microsoft.Bot.Builder.AI.QnA.Metadata();
150+
var qnaOptions = new QnAMakerOptions();
151+
152+
qnaOptions.Top = Constants.DefaultTop;
153+
qnaOptions.ScoreThreshold = 0.3F;
154+
var response = await _services.QnAServices[QnAMakerKey].GetAnswersAsync(turnContext, qnaOptions);
155+
```
156+
157+
The Support bot has [an example](https://github.com/microsoft/BotBuilder-Samples/blob/master/experimental/qnamaker-support/csharp_dotnetcore/Service/SupportBotService.cs#L418) with this code.
158+
159+
## Use QnA Maker with a bot in Node.js
160+
161+
The bot framework provides access to the QnA Maker's properties:
162+
163+
```javascript
164+
const { QnAMaker } = require('botbuilder-ai');
165+
this.qnaMaker = new QnAMaker(endpoint);
166+
167+
// Default QnAMakerOptions
168+
var qnaMakerOptions = {
169+
ScoreThreshold: 0.03,
170+
Top: 3
171+
};
172+
var qnaResults = await this.qnaMaker.getAnswers(stepContext.context, qnaMakerOptions);
173+
```
174+
175+
The Support bot has [an example](https://github.com/microsoft/BotBuilder-Samples/blob/master/experimental/qnamaker-activelearning/javascript_nodejs/Helpers/dialogHelper.js#L36) with this code.
176+
143177
<a name="metadata-example"></a>
144178

145179
## Use metadata to filter answers by custom metadata tags

0 commit comments

Comments
 (0)