You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cognitive-services/QnAMaker/How-To/improve-knowledge-base.md
+44-48Lines changed: 44 additions & 48 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
---
2
-
title: Improve knowledge base - QnA Maker
2
+
title: Improve your knowledge base - QnA Maker
3
3
titleSuffix: Azure Cognitive Services
4
-
description: Active learning allows you to improve the quality of your knowledge base by suggesting alternative questions, based on user-submissions, to your question and answer pair. You review those suggestions, either adding them to existing questions or rejecting them. Your knowledge base doesn't change automatically. You must accept the suggestions for any change to take effect. These suggestions add questions but don't change or remove existing questions.
4
+
description: Active learning allows you to improve the quality of your knowledge base by suggesting alternative questions, based on usersubmissions, to your question and answer pair. You review those suggestions, either adding them to existing questions or rejecting them.
5
5
author: diberry
6
6
manager: nitinme
7
7
services: cognitive-services
@@ -14,89 +14,83 @@ ms.author: diberry
14
14
15
15
# Use active learning to improve your knowledge base
16
16
17
-
Active learningallows you to improve the quality of your knowledge base by suggesting alternative questions, based on user-submissions, to your question and answer pair. You review those suggestions, either adding them to existing questions or rejecting them.
17
+
QnA Maker features *active learning*. This capability allows you to improve the quality of your knowledge base by suggesting alternative questions, based on usersubmissions, to your question and answer pair. You review those suggestions, either adding them to existing questions or rejecting them.
18
18
19
-
Your knowledge base doesn't change automatically. You must accept the suggestions in order for any change to take effect. These suggestions add questions but don't change or remove existing questions.
19
+
Your knowledge base doesn't change automatically. You must accept the suggestions for any change to take effect. These suggestions add questions, but don't change or remove existing questions.
20
20
21
21
## What is active learning?
22
22
23
23
QnA Maker learns new question variations with implicit and explicit feedback.
24
24
25
-
* Implicit feedback – The ranker understands when a user question has multiple answers with scores that are very close and considers this as feedback.
26
-
* Explicit feedback – When multiple answers with little variation in scores are returned from the knowledge base, the client application asks the user which question is the correct question. The user's explicit feedback is sent to QnA Maker with the Train API.
25
+
***Implicit feedback.** When a user question has multiple answers with scores that are very close, the QnA Maker ranker considers this as feedback.
26
+
***Explicit feedback.**When the knowledge base returns multiple answers with little variation in scores, the client application asks the user which question is the correct question. The user's explicit feedback is sent to QnA Maker with the Train API.
27
27
28
28
Either method provides the ranker with similar queries that are clustered.
29
29
30
30
## How active learning works
31
31
32
-
Active learning is triggered based on the scores of top few answers returned by QnA Maker for any given query. If the score differences lie within a small range, then the query is considered a possible _suggestion_ for each of the possible answers.
32
+
For any specified query, active learning is triggered based on the scores of top few answers returned by QnA Maker. If the score differences lie within a small range, then the query is considered a possible _suggestion_ for each of the possible answers.
33
33
34
-
All the suggestions are clustered together by similarity and top suggestions for alternate questions are displayed based on the frequency of the particular queries by end users. Active learning gives the best possible suggestions in cases where the endpoints are getting a reasonable quantity and variety of usage queries.
34
+
QnA Maker clusters all the suggestions together by similarity. It displays top suggestions for alternate questions, based on the frequency of the particular queries by end users. Active learning gives the best possible suggestions in cases where the endpoints are getting a reasonable quantity and variety of usage queries.
35
35
36
-
When 5 or more similar queries are clustered, every 30 minutes, QnA Maker suggests the user-based questions to the knowledge base designer to accept or reject.
37
-
38
-
Once questions are suggested in the QnA Maker portal, you need to review and accept or reject those suggestions.
36
+
When 5 or more similar queries are clustered, every 30 minutes, QnA Maker suggests the user-based questions to the knowledge base designer to accept or reject. You need to review and accept or reject those suggestions.
39
37
40
38
## Upgrade your version to use active learning
41
39
42
-
Active Learning is supported in runtime version 4.4.0 and above. If your knowledge base was created on an earlier version, [upgrade your runtime](troubleshooting-runtime.md#how-to-get-latest-qnamaker-runtime-updates) to use this feature.
43
-
44
-
## Best practices
45
-
46
-
For best practices when using active learning, see [Best practices](../Concepts/best-practices.md#active-learning).
40
+
QnA Maker supports active learning in runtime version 4.4.0 and later. If your knowledge base was created on an earlier version, [upgrade your runtime](troubleshooting-runtime.md#how-to-get-latest-qnamaker-runtime-updates) to use this feature.
47
41
48
42
## Score proximity between knowledge base questions
49
43
50
-
When a question's score is highly confident, such as 80%, the range of scores that are considered for active learning are wide, approximately within 10%. As the confidence score decreases, such as 40%, the range of scores decreases as well, approximately within 4%.
44
+
When a question's score is highly confident, such as 80 percent, the range of scores that are considered for active learning are wide, approximately within 10 percent. As the confidence score decreases, such as 40 percent, the range of scores decreases as well, approximately within 4 percent.
51
45
52
-
The algorithm to determine proximity is not a simple calculation. The ranges in the preceding examples are not meant to be fixed but should be used as a guide to understand the impact of the algorithm only.
46
+
The algorithm to determine proximity is not a simple calculation. The ranges in the preceding examples are not meant to be fixed, but you can use them as a guide to understand the impact of the algorithm.
53
47
54
48
## Turn on active learning
55
49
56
50
Active learning is off by default. Turn it on to see suggested questions.
57
51
58
-
1. Select **Publish** to publish the knowledge base. Active learning queries are collected from the GenerateAnswer API prediction endpoint only. The queries to the Test pane in the QnA Maker portal do not impact active learning.
52
+
1. Select **Publish** to publish the knowledge base. Active learning queries are collected from the GenerateAnswer API prediction endpoint only. The queries to the Test pane in the QnA Maker portal don't impact active learning.
59
53
60
-
1. To turn active learning on, click on your **Name**, go to [**Service Settings**](https://www.qnamaker.ai/UserSettings) in the QnA Maker portal, in the top right corner.
54
+
1. To turn active learning on, select your **Name**, and go to [**Service Settings**](https://www.qnamaker.ai/UserSettings) in the QnA Maker portal, in the top right corner.
61
55
62
-
56
+
63
57
64
58
65
-
1. Find the QnA Maker servicethen toggle **Active Learning**.
59
+
1. Find the QnA Maker service, and then toggle to**Active Learning**.
66
60
67
-
[](../media/improve-knowledge-base/turn-active-learning-on-at-service-setting.png#lightbox)
61
+
[
68
62
69
-
Once **Active Learning** is enabled, the knowledge suggests new questions at regular intervals based on user-submitted questions. You can disable **Active Learning** by toggling the setting again.
63
+
When active learning is enabled, the knowledge base suggests new questions at regular intervals based on user-submitted questions. You can disable active learning by toggling the setting again.
70
64
71
-
## Add active learning suggestion to knowledge base
65
+
## Add an active learning suggestion to the knowledge base
72
66
73
-
1.In order to see the suggested questions, on the **Edit** knowledge base page, select **View Options**, then select **Show active learning suggestions**.
67
+
1.To see the suggested questions, on the **Edit** knowledge base page, select **View Options**. Then select **Show active learning suggestions**.
74
68
75
-
[](../media/improve-knowledge-base/show-suggestions-button.png#lightbox)
69
+
[](../media/improve-knowledge-base/show-suggestions-button.png#lightbox)
76
70
77
-
1. Filter the knowledge base with question and answer pairs to show only suggestions by selecting **Filter by Suggestions**.
71
+
1. Filter the knowledge base with question and answer pairs to show only suggestions, by selecting **Filter by Suggestions**.
78
72
79
-
[](../media/improve-knowledge-base/filter-by-suggestions.png#lightbox)
73
+
[](../media/improve-knowledge-base/filter-by-suggestions.png#lightbox)
80
74
81
-
1. Each question section with suggestions shows the new questions with a check mark, `✔` , to accept the question or an `x` to reject the suggestions. Select the check mark to add the question.
75
+
1. Each question section with suggestions shows the new questions with either a check mark, `✔` , to accept the question, or an `x` to reject the suggestions. Select the check mark to add the question.
82
76
83
-
[](../media/improve-knowledge-base/accept-active-learning-suggestions.png#lightbox)
77
+
[](../media/improve-knowledge-base/accept-active-learning-suggestions.png#lightbox)
84
78
85
-
You can add or delete _all suggestions_ by selecting **Add all** or **Reject all**.
79
+
You can add or delete all suggestions by selecting **Add all** or **Reject all**.
86
80
87
81
1. Select **Save and Train** to save the changes to the knowledge base.
88
82
89
83
1. Select **Publish** to allow the changes to be available from the GenerateAnswer API.
90
84
91
85
When 5 or more similar queries are clustered, every 30 minutes, QnA Maker suggests the user-based questions to the knowledge base designer to accept or reject.
92
86
93
-
## Determine best choice when several questions have similar scores
87
+
## Ask for clarification
94
88
95
-
When a question is too close in score to other questions, the client-application developer can choose to ask for clarification.
89
+
When a question is too close in score to other questions, you can ask QnA Maker for clarification.
96
90
97
91
### Use the top property in the GenerateAnswer request
98
92
99
-
When submitting a question to QnA Maker for an answer, part of the JSON body allows for returning more than one top answer:
93
+
When submitting a question to QnA Maker for an answer, you can use the top property of the JSON body, to return more than one top answer:
100
94
101
95
```json
102
96
{
@@ -106,7 +100,7 @@ When submitting a question to QnA Maker for an answer, part of the JSON body all
106
100
}
107
101
```
108
102
109
-
When the client application (such as a chat bot) receives the response, the top 3 questions are returned:
103
+
In the preceding example, the client application (such as a chat bot) receives a response that includes the top 3 questions:
110
104
111
105
```json
112
106
{
@@ -147,15 +141,15 @@ When the client application (such as a chat bot) receives the response, the top
147
141
148
142
### Client application follow-up when questions have similar scores
149
143
150
-
The client application displays all the questions with an option for the user to select the question that most represents their intention.
144
+
The client application displays all the questions, with an option for the user to select the question that most represents their intention.
151
145
152
-
Once the user selects one of the existing questions, the client application sends the user's choice as feedback using QnA Maker's Train API. This feedback completes the active learning feedback loop.
146
+
When the user selects one of the existing questions, the client application sends the user's choice as feedback by using QnA Maker's Train API. This feedback completes the active learning feedback loop.
153
147
154
148
Use the [Azure Bot sample](https://aka.ms/activelearningsamplebot) to see active learning in an end-to-end scenario.
155
149
156
150
## Train API
157
151
158
-
Active learning feedback is sent to QnA Maker with the Train API POST request. The API signature is:
152
+
The Train API POST request sends active learning feedback to QnA Maker. The API signature is:
159
153
160
154
```http
161
155
POST https://<QnA-Maker-resource-name>.azurewebsites.net/qnamaker/knowledgebases/<knowledge-base-ID>/train
|URL route parameter|Knowledge base ID|string|The GUID for your knowledge base.|
170
-
|Host subdomain|QnAMaker resource name|string|The hostname for your QnA Maker in your Azure subscription. This is available on the Settings page after you publish the knowledge base. |
171
-
|Header|Content-Type|string|The media type of the body sent to the API. Default value is: `application/json`|
164
+
|Host subdomain|QnAMaker resource name|string|The hostname for your QnA Maker in your Azure subscription. This is available on the **Settings** page after you publish the knowledge base. |
165
+
|Header|Content-Type|string|The media type of the body sent to the API. The default value is: `application/json`.|
|Post Body|JSON object|JSON|The training feedback|
167
+
|Post body|JSON object|JSON|The training feedback.|
174
168
175
169
The JSON body has several settings:
176
170
177
171
|JSON body property|Type|Purpose|
178
172
|--|--|--|--|
179
173
|`feedbackRecords`|array|List of feedback.|
180
-
|`userId`|string|The user ID of the person accepting the suggested questions. The user ID format is up to you. For example, an email address can be a valid user ID in your architecture. Optional.|
181
-
|`userQuestion`|string|Exact text of the question. Required.|
182
-
|`qnaID`|number|ID of question, found in the [GenerateAnswer response](metadata-generateanswer-usage.md#generateanswer-response-properties). |
174
+
|`userId`|string|The user ID of the person accepting the suggested questions. The user ID format is up to you. For example, an email address can be a valid user ID in your architecture. This property is optional.|
175
+
|`userQuestion`|string|Exact text of the question. This property is required.|
176
+
|`qnaID`|number|The ID of the question, found in the [GenerateAnswer response](metadata-generateanswer-usage.md#generateanswer-response-properties). |
183
177
184
178
An example JSON body looks like:
185
179
@@ -195,13 +189,13 @@ An example JSON body looks like:
195
189
}
196
190
```
197
191
198
-
A successful response returns a status of 204 and no JSON response body.
192
+
A successful response returns a status of 204, and no JSON response body.
## Active learning is saved in the exported knowledge base
203
197
204
-
When your app has active learning enabled, and you export the app, the `SuggestedQuestions` column in the tsv file retains the active learning data.
198
+
When your app has active learning enabled, and you export the app, the `SuggestedQuestions` column in the .tsv file retains the active learning data.
205
199
206
200
The `SuggestedQuestions` column is a JSON object of information of implicit, `autosuggested`, and explicit, `usersuggested` feedback. An example of this JSON object for a single user-submitted question of `help` is:
207
201
@@ -225,6 +219,8 @@ The `SuggestedQuestions` column is a JSON object of information of implicit, `au
225
219
When you reimport this app, the active learning continues to collect information and recommend suggestions for your knowledge base.
226
220
227
221
## Next steps
228
-
222
+
223
+
For best practices when using active learning, see [Best practices](../Concepts/best-practices.md#active-learning).
224
+
229
225
> [!div class="nextstepaction"]
230
226
> [Use metadata with GenerateAnswer API](metadata-generateanswer-usage.md)
0 commit comments