You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-studio/how-to/develop/evaluate-sdk.md
+11-11Lines changed: 11 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -122,7 +122,7 @@ For evaluators that support conversations, you can provide `conversation` as inp
122
122
}
123
123
```
124
124
125
-
Our evaluators will understand that the first turn of the conversation provides valid `query` from `user`, `context` from `assistant`, and `response` from `assistant` in the query-response format. Conversations are then evaluated per turn and results are aggregated over all turns for a conversation score.
125
+
Our evaluators understand that the first turn of the conversation provides valid `query` from `user`, `context` from `assistant`, and `response` from `assistant` in the query-response format. Conversations are then evaluated per turn and results are aggregated over all turns for a conversation score.
126
126
127
127
> [!NOTE]
128
128
> Note that in the second turn, even if `context` is `null` or a missing key, it will be interpreted as an empty string instead of erroring out, which might lead to misleading results. We strongly recommend that you validate your evaluation data to comply with the data requirements.
@@ -500,7 +500,7 @@ After you spot-check your built-in or custom evaluators on a single row of data,
500
500
501
501
If you want to enable logging and tracing to your Azure AI project for evaluation results, follow these steps:
502
502
503
-
1. Make sure you are first logged in by running `az login`.
503
+
1. Make sure you're first logged in by running `az login`.
4. Make sure you have `Storage Blob Data Contributor` role for the storage account.
512
512
513
-
### Local evaluaton on datasets
513
+
### Local evaluation on datasets
514
514
In order to ensure the `evaluate()` can correctly parse the data, you must specify column mapping to map the column from the dataset to key words that are accepted by the evaluators. In this case, we specify the data mapping for `query`, `response`, and `context`.
515
515
516
516
```python
@@ -669,7 +669,7 @@ result = evaluate(
669
669
670
670
## Cloud evaluation on test datasets
671
671
672
-
After local evaluations of your generative AI applications, you may want to run evaluations in the cloud for pre-deployment testing and [continuously evaluate](https://aka.ms/GenAIMonitoringDoc) your applications for post-deployment monitoring. Azure AI Projects SDK offers such capabilities via a Python API and supports almost all of the features available in local evaluations. Follow the steps below to submit your evaluation to the cloud on your data using built-in or custom evaluators.
672
+
After local evaluations of your generative AI applications, you may want to run evaluations in the cloud for pre-deployment testing, and [continuously evaluate](https://aka.ms/GenAIMonitoringDoc) your applications for post-deployment monitoring. Azure AI Projects SDK offers such capabilities via a Python API and supports almost all of the features available in local evaluations. Follow the steps below to submit your evaluation to the cloud on your data using built-in or custom evaluators.
673
673
674
674
675
675
### Prerequisites
@@ -680,7 +680,7 @@ After local evaluations of your generative AI applications, you may want to run
680
680
681
681
- Azure OpenAI Deployment with GPT model supporting `chat completion`, for example `gpt-4`.
682
682
-`Connection String` for Azure AI project to easily create `AIProjectClient` object. You can get the **Project connection string** under **Project details** from the project's **Overview** page.
683
-
- Make sure you are first logged into your Azure subscription by running `az login`.
683
+
- Make sure you're first logged into your Azure subscription by running `az login`.
684
684
685
685
### Installation Instructions
686
686
@@ -693,7 +693,7 @@ After local evaluations of your generative AI applications, you may want to run
**From UI**: Alternatively, you can upload new data or update existing data versions by following the UI walkthrough under the **Data** tab of your Azure AI project.
725
725
726
726
2. Given existing datasets uploaded to your Project:
727
-
- **From SDK**: if you already know the dataset name you created, construct the dataset idin this format: `/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.MachineLearningServices/workspaces/<project-name>/data/<dataset-name>/versions/<version-number>`
727
+
- **From SDK**: if you already know the dataset name you created, construct the dataset IDin this format: `/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.MachineLearningServices/workspaces/<project-name>/data/<dataset-name>/versions/<version-number>`
728
728
729
-
- **From UI**: If you don't know the dataset name, locate it under the **Data** tab of your Azure AI project and construct the dataset id as in the format above.
729
+
- **From UI**: If you don't know the dataset name, locate it under the **Data** tab of your Azure AI project and construct the dataset ID as in the format above.
0 commit comments