Skip to content

Commit 95b8cfe

Browse files
Merge pull request #273 from peytonfraser/patch-3
Update best-practices.md for "AIQ" and "recipe"
2 parents d66bb93 + 6b8589c commit 95b8cfe

File tree

1 file changed

+6
-6
lines changed
  • articles/ai-services/language-service/conversational-language-understanding/concepts

1 file changed

+6
-6
lines changed

articles/ai-services/language-service/conversational-language-understanding/concepts/best-practices.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ You also want to avoid mixing different schema designs. Don't build half of your
4747

4848
## Use standard training before advanced training
4949

50-
[Standard training](../how-to/train-model.md#training-modes) is free and faster than advanced training. It can help you quickly understand the effect of changing your training set or schema while you build the model. After you're satisfied with the schema, consider using advanced training to get the best AIQ out of your model.
50+
[Standard training](../how-to/train-model.md#training-modes) is free and faster than advanced training. It can help you quickly understand the effect of changing your training set or schema while you build the model. After you're satisfied with the schema, consider using advanced training to get the best model quality.
5151

5252
## Use the evaluation feature
5353

@@ -113,7 +113,7 @@ If you enable this feature, the utterance count of your training set increases.
113113

114114
## Address model overconfidence
115115

116-
Customers can use the LoraNorm recipe version if the model is being incorrectly overconfident. An example of this behavior can be like the following scenario where the model predicts the incorrect intent with 100% confidence. This score makes the confidence threshold project setting unusable.
116+
Customers can use the LoraNorm traning configuration version if the model is being incorrectly overconfident. An example of this behavior can be like the following scenario where the model predicts the incorrect intent with 100% confidence. This score makes the confidence threshold project setting unusable.
117117

118118
| Text | Predicted intent | Confidence score |
119119
|----|----|----|
@@ -243,7 +243,7 @@ curl --request POST \
243243

244244
## Address out-of-domain utterances
245245

246-
Customers can use the newly updated recipe version `2024-08-01-preview` (previously `2024-06-01-preview`) if the model has poor AIQ on out-of-domain utterances. An example of this scenario with the default recipe can be like the following example where the model has three intents: `Sports`, `QueryWeather`, and `Alarm`. The test utterances are out-of-domain utterances and the model classifies them as `InDomain` with a relatively high confidence score.
246+
Customers can use the newly updated training configuration version `2024-08-01-preview` (previously `2024-06-01-preview`) if the model has poor quality on out-of-domain utterances. An example of this scenario with the default training configuration can be like the following example where the model has three intents: `Sports`, `QueryWeather`, and `Alarm`. The test utterances are out-of-domain utterances and the model classifies them as `InDomain` with a relatively high confidence score.
247247

248248
| Text | Predicted intent | Confidence score |
249249
|----|----|----|
@@ -273,6 +273,6 @@ After the request is sent, you can track the progress of the training job in Lan
273273

274274
Caveats:
275275

276-
- The None score threshold for the app (confidence threshold below which `topIntent` is marked as `None`) when you use this recipe should be set to 0. This setting is used because this new recipe attributes a certain portion of the in-domain probabilities to out of domain so that the model isn't incorrectly overconfident about in-domain utterances. As a result, users might see slightly reduced confidence scores for in-domain utterances as compared to the prod recipe.
277-
- We don't recommend this recipe for apps with only two intents, such as `IntentA` and `None`, for example.
278-
- We don't recommend this recipe for apps with a low number of utterances per intent. We highly recommend a minimum of 25 utterances per intent.
276+
- The None score threshold for the app (confidence threshold below which `topIntent` is marked as `None`) when you use this training configuration should be set to 0. This setting is used because this new training configuration attributes a certain portion of the in-domain probabilities to out of domain so that the model isn't incorrectly overconfident about in-domain utterances. As a result, users might see slightly reduced confidence scores for in-domain utterances as compared to the prod training configuration.
277+
- We don't recommend this training configuration for apps with only two intents, such as `IntentA` and `None`, for example.
278+
- We don't recommend this training configuration for apps with a low number of utterances per intent. We highly recommend a minimum of 25 utterances per intent.

0 commit comments

Comments
 (0)