You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-services/language-service/conversational-language-understanding/concepts/best-practices.md
+35Lines changed: 35 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -216,3 +216,38 @@ curl --request POST \
216
216
"targetResourceRegion": "<target-region>"
217
217
}'
218
218
```
219
+
220
+
221
+
## Addressing Out of Domain utterances
222
+
223
+
Customers can use the new recipe version '2024-06-01-preview' in case the model has poor AIQ on out of domain utterances. An example of this with the default recipe can be like the below where the model has 3 intents Sports, QueryWeather and Alarm. The test utterances are out of domain utterances and the model classifies them as InDomain with a relatively high confidence score.
224
+
225
+
| Text | Predicted intent | Confidence score |
226
+
|----|----|----|
227
+
| "*Who built the Eiffel Tower?*" |`Sports`| 0.90 |
228
+
| "*Do I look good to you today?*" |`QueryWeather`| 1.00 |
229
+
| "*I hope you have a good evening.*" |`Alarm`| 0.80 |
230
+
231
+
To address this, use the `2024-06-01-preview` configuration version that is built specifically to address this issue while also maintaining reasonably good quality on In Domain utterances.
Once the request is sent, you can track the progress of the training job in Language Studio as usual.
250
+
251
+
Caveats:
252
+
- The None Score threshold for the app (confidence threshold below which the topIntent is marked as None) when using this recipe should be set to 0. This is because this new recipe attributes a certain portion of the in domain probabiliities to out of domain so that the model is not incorrectly overconfident about in domain utterances. As a result, users may see slightly reduced confidence scores for in domain utterances as compared to the prod recipe.
253
+
- This recipe is not recommended for apps with just 2 intents, eg: IntentA and None.
0 commit comments