You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cognitive-services/QnAMaker/How-To/improve-knowledge-base.md
+48-44Lines changed: 48 additions & 44 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
---
2
-
title: Improve your knowledge base - QnA Maker
2
+
title: Improve knowledge base - QnA Maker
3
3
titleSuffix: Azure Cognitive Services
4
-
description: Active learning allows you to improve the quality of your knowledge base by suggesting alternative questions, based on usersubmissions, to your question and answer pair. You review those suggestions, either adding them to existing questions or rejecting them.
4
+
description: Active learning allows you to improve the quality of your knowledge base by suggesting alternative questions, based on user-submissions, to your question and answer pair. You review those suggestions, either adding them to existing questions or rejecting them. Your knowledge base doesn't change automatically. You must accept the suggestions for any change to take effect. These suggestions add questions but don't change or remove existing questions.
5
5
author: diberry
6
6
manager: nitinme
7
7
services: cognitive-services
@@ -14,83 +14,89 @@ ms.author: diberry
14
14
15
15
# Use active learning to improve your knowledge base
16
16
17
-
QnA Maker features *active learning*. This capability allows you to improve the quality of your knowledge base by suggesting alternative questions, based on usersubmissions, to your question and answer pair. You review those suggestions, either adding them to existing questions or rejecting them.
17
+
Active learningallows you to improve the quality of your knowledge base by suggesting alternative questions, based on user-submissions, to your question and answer pair. You review those suggestions, either adding them to existing questions or rejecting them.
18
18
19
-
Your knowledge base doesn't change automatically. You must accept the suggestions for any change to take effect. These suggestions add questions, but don't change or remove existing questions.
19
+
Your knowledge base doesn't change automatically. You must accept the suggestions in order for any change to take effect. These suggestions add questions but don't change or remove existing questions.
20
20
21
21
## What is active learning?
22
22
23
23
QnA Maker learns new question variations with implicit and explicit feedback.
24
24
25
-
***Implicit feedback.** When a user question has multiple answers with scores that are very close, the QnA Maker ranker considers this as feedback.
26
-
***Explicit feedback.**When the knowledge base returns multiple answers with little variation in scores, the client application asks the user which question is the correct question. The user's explicit feedback is sent to QnA Maker with the Train API.
25
+
* Implicit feedback – The ranker understands when a user question has multiple answers with scores that are very close and considers this as feedback.
26
+
* Explicit feedback – When multiple answers with little variation in scores are returned from the knowledge base, the client application asks the user which question is the correct question. The user's explicit feedback is sent to QnA Maker with the Train API.
27
27
28
28
Either method provides the ranker with similar queries that are clustered.
29
29
30
30
## How active learning works
31
31
32
-
For any specified query, active learning is triggered based on the scores of top few answers returned by QnA Maker. If the score differences lie within a small range, then the query is considered a possible _suggestion_ for each of the possible answers.
32
+
Active learning is triggered based on the scores of top few answers returned by QnA Maker for any given query. If the score differences lie within a small range, then the query is considered a possible _suggestion_ for each of the possible answers.
33
33
34
-
QnA Maker clusters all the suggestions together by similarity. It displays top suggestions for alternate questions, based on the frequency of the particular queries by end users. Active learning gives the best possible suggestions in cases where the endpoints are getting a reasonable quantity and variety of usage queries.
34
+
All the suggestions are clustered together by similarity and top suggestions for alternate questions are displayed based on the frequency of the particular queries by end users. Active learning gives the best possible suggestions in cases where the endpoints are getting a reasonable quantity and variety of usage queries.
35
35
36
-
When 5 or more similar queries are clustered, every 30 minutes, QnA Maker suggests the user-based questions to the knowledge base designer to accept or reject. You need to review and accept or reject those suggestions.
36
+
When 5 or more similar queries are clustered, every 30 minutes, QnA Maker suggests the user-based questions to the knowledge base designer to accept or reject.
37
+
38
+
Once questions are suggested in the QnA Maker portal, you need to review and accept or reject those suggestions.
37
39
38
40
## Upgrade your version to use active learning
39
41
40
-
QnA Maker supports active learning in runtime version 4.4.0 and later. If your knowledge base was created on an earlier version, [upgrade your runtime](troubleshooting-runtime.md#how-to-get-latest-qnamaker-runtime-updates) to use this feature.
42
+
Active Learning is supported in runtime version 4.4.0 and above. If your knowledge base was created on an earlier version, [upgrade your runtime](troubleshooting-runtime.md#how-to-get-latest-qnamaker-runtime-updates) to use this feature.
43
+
44
+
## Best practices
45
+
46
+
For best practices when using active learning, see [Best practices](../Concepts/best-practices.md#active-learning).
41
47
42
48
## Score proximity between knowledge base questions
43
49
44
-
When a question's score is highly confident, such as 80 percent, the range of scores that are considered for active learning are wide, approximately within 10 percent. As the confidence score decreases, such as 40 percent, the range of scores decreases as well, approximately within 4 percent.
50
+
When a question's score is highly confident, such as 80%, the range of scores that are considered for active learning are wide, approximately within 10%. As the confidence score decreases, such as 40%, the range of scores decreases as well, approximately within 4%.
45
51
46
-
The algorithm to determine proximity is not a simple calculation. The ranges in the preceding examples are not meant to be fixed, but you can use them as a guide to understand the impact of the algorithm.
52
+
The algorithm to determine proximity is not a simple calculation. The ranges in the preceding examples are not meant to be fixed but should be used as a guide to understand the impact of the algorithm only.
47
53
48
54
## Turn on active learning
49
55
50
56
Active learning is off by default. Turn it on to see suggested questions.
51
57
52
-
1. Select **Publish** to publish the knowledge base. Active learning queries are collected from the GenerateAnswer API prediction endpoint only. The queries to the Test pane in the QnA Maker portal don't affect active learning.
58
+
1. Select **Publish** to publish the knowledge base. Active learning queries are collected from the GenerateAnswer API prediction endpoint only. The queries to the Test pane in the QnA Maker portal do not impact active learning.
53
59
54
-
1. To turn active learning on, select your **Name**, and go to [**Service Settings**](https://www.qnamaker.ai/UserSettings) in the QnA Maker portal, in the top right corner.
60
+
1. To turn active learning on, click on your **Name**, go to [**Service Settings**](https://www.qnamaker.ai/UserSettings) in the QnA Maker portal, in the top right corner.
55
61
56
-
62
+
57
63
58
64
59
-
1. Find the QnA Maker service, and then toggle to**Active Learning**.
65
+
1. Find the QnA Maker servicethen toggle **Active Learning**.
60
66
61
-
[](../media/improve-knowledge-base/turn-active-learning-on-at-service-setting.png#lightbox)
67
+
[](../media/improve-knowledge-base/turn-active-learning-on-at-service-setting.png#lightbox)
62
68
63
-
When active learning is enabled, the knowledge base suggests new questions at regular intervals based on user-submitted questions. You can disable active learning by toggling the setting again.
69
+
Once **Active Learning** is enabled, the knowledge suggests new questions at regular intervals based on user-submitted questions. You can disable **Active Learning** by toggling the setting again.
64
70
65
-
## Add an active learning suggestion to the knowledge base
71
+
## Add active learning suggestion to knowledge base
66
72
67
-
1.To see the suggested questions, on the **Edit** knowledge base page, select **View Options**. Then select **Show active learning suggestions**.
73
+
1.In order to see the suggested questions, on the **Edit** knowledge base page, select **View Options**, then select **Show active learning suggestions**.
68
74
69
-
[](../media/improve-knowledge-base/show-suggestions-button.png#lightbox)
75
+
[](../media/improve-knowledge-base/show-suggestions-button.png#lightbox)
70
76
71
-
1. Filter the knowledge base with question and answer pairs to show only suggestions, by selecting **Filter by Suggestions**.
77
+
1. Filter the knowledge base with question and answer pairs to show only suggestions by selecting **Filter by Suggestions**.
72
78
73
-
[](../media/improve-knowledge-base/filter-by-suggestions.png#lightbox)
79
+
[](../media/improve-knowledge-base/filter-by-suggestions.png#lightbox)
74
80
75
-
1. Each question section with suggestions shows the new questions with either a check mark, `✔` , to accept the question, or an `x` to reject the suggestions. Select the check mark to add the question.
81
+
1. Each question section with suggestions shows the new questions with a check mark, `✔` , to accept the question or an `x` to reject the suggestions. Select the check mark to add the question.
76
82
77
-
[](../media/improve-knowledge-base/accept-active-learning-suggestions.png#lightbox)
83
+
[](../media/improve-knowledge-base/accept-active-learning-suggestions.png#lightbox)
78
84
79
-
You can add or delete all suggestions by selecting **Add all** or **Reject all**.
85
+
You can add or delete _all suggestions_ by selecting **Add all** or **Reject all**.
80
86
81
87
1. Select **Save and Train** to save the changes to the knowledge base.
82
88
83
89
1. Select **Publish** to allow the changes to be available from the GenerateAnswer API.
84
90
85
91
When 5 or more similar queries are clustered, every 30 minutes, QnA Maker suggests the user-based questions to the knowledge base designer to accept or reject.
86
92
87
-
## Ask for clarification
93
+
## Determine best choice when several questions have similar scores
88
94
89
-
When a question is too close in score to other questions, you can ask QnA Maker for clarification.
95
+
When a question is too close in score to other questions, the client-application developer can choose to ask for clarification.
90
96
91
97
### Use the top property in the GenerateAnswer request
92
98
93
-
When submitting a question to QnA Maker for an answer, you can use the top property of the JSON body, to return more than one top answer:
99
+
When submitting a question to QnA Maker for an answer, part of the JSON body allows for returning more than one top answer:
94
100
95
101
```json
96
102
{
@@ -100,7 +106,7 @@ When submitting a question to QnA Maker for an answer, you can use the top prope
100
106
}
101
107
```
102
108
103
-
In the preceding example, the client application (such as a chat bot) receives a response that includes the top 3 questions:
109
+
When the client application (such as a chat bot) receives the response, the top 3 questions are returned:
104
110
105
111
```json
106
112
{
@@ -141,15 +147,15 @@ In the preceding example, the client application (such as a chat bot) receives a
141
147
142
148
### Client application follow-up when questions have similar scores
143
149
144
-
The client application displays all the questions, with an option for the user to select the question that most represents their intention.
150
+
The client application displays all the questions with an option for the user to select the question that most represents their intention.
145
151
146
-
When the user selects one of the existing questions, the client application sends the user's choice as feedback by using QnA Maker's Train API. This feedback completes the active learning feedback loop.
152
+
Once the user selects one of the existing questions, the client application sends the user's choice as feedback using QnA Maker's Train API. This feedback completes the active learning feedback loop.
147
153
148
154
Use the [Azure Bot sample](https://aka.ms/activelearningsamplebot) to see active learning in an end-to-end scenario.
149
155
150
156
## Train API
151
157
152
-
The Train API POST request sends active learning feedback to QnA Maker. The API signature is:
158
+
Active learning feedback is sent to QnA Maker with the Train API POST request. The API signature is:
153
159
154
160
```http
155
161
POST https://<QnA-Maker-resource-name>.azurewebsites.net/qnamaker/knowledgebases/<knowledge-base-ID>/train
|URL route parameter|Knowledge base ID|string|The GUID for your knowledge base.|
164
-
|Host subdomain|QnAMaker resource name|string|The hostname for your QnA Maker in your Azure subscription. This is available on the **Settings** page after you publish the knowledge base. |
165
-
|Header|Content-Type|string|The media type of the body sent to the API. The default value is: `application/json`.|
170
+
|Host subdomain|QnAMaker resource name|string|The hostname for your QnA Maker in your Azure subscription. This is available on the Settings page after you publish the knowledge base. |
171
+
|Header|Content-Type|string|The media type of the body sent to the API. Default value is: `application/json`|
|Post body|JSON object|JSON|The training feedback.|
173
+
|Post Body|JSON object|JSON|The training feedback|
168
174
169
175
The JSON body has several settings:
170
176
171
177
|JSON body property|Type|Purpose|
172
178
|--|--|--|--|
173
179
|`feedbackRecords`|array|List of feedback.|
174
-
|`userId`|string|The user ID of the person accepting the suggested questions. The user ID format is up to you. For example, an email address can be a valid user ID in your architecture. This property is optional.|
175
-
|`userQuestion`|string|Exact text of the question. This property is required.|
176
-
|`qnaID`|number|The ID of the question, found in the [GenerateAnswer response](metadata-generateanswer-usage.md#generateanswer-response-properties). |
180
+
|`userId`|string|The user ID of the person accepting the suggested questions. The user ID format is up to you. For example, an email address can be a valid user ID in your architecture. Optional.|
181
+
|`userQuestion`|string|Exact text of the question. Required.|
182
+
|`qnaID`|number|ID of question, found in the [GenerateAnswer response](metadata-generateanswer-usage.md#generateanswer-response-properties). |
177
183
178
184
An example JSON body looks like:
179
185
@@ -189,13 +195,13 @@ An example JSON body looks like:
189
195
}
190
196
```
191
197
192
-
A successful response returns a status of 204, and no JSON response body.
198
+
A successful response returns a status of 204 and no JSON response body.
## Active learning is saved in the exported knowledge base
197
203
198
-
When your app has active learning enabled, and you export the app, the `SuggestedQuestions` column in the .tsv file retains the active learning data.
204
+
When your app has active learning enabled, and you export the app, the `SuggestedQuestions` column in the tsv file retains the active learning data.
199
205
200
206
The `SuggestedQuestions` column is a JSON object of information of implicit, `autosuggested`, and explicit, `usersuggested` feedback. An example of this JSON object for a single user-submitted question of `help` is:
201
207
@@ -219,8 +225,6 @@ The `SuggestedQuestions` column is a JSON object of information of implicit, `au
219
225
When you reimport this app, the active learning continues to collect information and recommend suggestions for your knowledge base.
220
226
221
227
## Next steps
222
-
223
-
For best practices when using active learning, see [Best practices](../Concepts/best-practices.md#active-learning).
224
-
228
+
225
229
> [!div class="nextstepaction"]
226
230
> [Use metadata with GenerateAnswer API](metadata-generateanswer-usage.md)
0 commit comments