Merge pull request #475 from MicrosoftDocs/main

9/24/2024 PM Publish

Author: Taojunshen

articles/ai-studio/how-to/develop/evaluate-sdk.md

@@ -9,9 +9,9 @@ ms.custom:
   - references_regions
 ms.topic: how-to
 ms.date: 09/24/2024
-ms.reviewer: dantaylo
-ms.author: eur
-author: eric-urban
+ms.reviewer: minthigpen
+ms.author: lagayhar
+author: lgayhardt
 ---
 # Evaluate with the Azure AI Evaluation SDK
 
@@ -87,17 +87,16 @@ When using AI-assisted performance and quality metrics, you must specify a GPT m
 You can run the built-in evaluators by importing the desired evaluator class. Ensure that you set your environment variables.
 ```python
 import os
-from promptflow.core import AzureOpenAIModelConfiguration
 
 # Initialize Azure OpenAI Connection with your environment variables
-model_config = AzureOpenAIModelConfiguration(
-    azure_endpoint=os.environ.get("AZURE_OPENAI_ENDPOINT"),
-    api_key=os.environ.get("AZURE_OPENAI_API_KEY"),
-    azure_deployment=os.environ.get("AZURE_OPENAI_DEPLOYMENT"),
-    api_version=os.environ.get("AZURE_OPENAI_API_VERSION"),
-)
+model_config = {
+    "azure_endpoint": os.environ.get("AZURE_OPENAI_ENDPOINT"),
+    "api_key": os.environ.get("AZURE_OPENAI_API_KEY"),
+    "azure_deployment": os.environ.get("AZURE_OPENAI_DEPLOYMENT"),
+    "api_version": os.environ.get("AZURE_OPENAI_API_VERSION"),
+}
 
-from azure.ai.evaluation.evaluators import RelevanceEvaluator
+from azure.ai.evaluation import RelevanceEvaluator
 
 # Initialzing Relevance Evaluator
 relevance_eval = RelevanceEvaluator(model_config)
@@ -131,7 +130,7 @@ azure_ai_project = {
     "project_name": "<project_name>",
 }
 
-from azure.ai.evaluation.evaluators import ViolenceEvaluator
+from azure.ai.evaluation import ViolenceEvaluator
 
 # Initializing Violence Evaluator with project information
 violence_eval = ViolenceEvaluator(azure_ai_project)
@@ -329,7 +328,7 @@ After logging your custom evaluator to your AI Studio project, you can view it i
 After you spot-check your built-in or custom evaluators on a single row of data, you can combine multiple evaluators with the `evaluate()` API on an entire test dataset. In order to ensure the `evaluate()` can correctly parse the data, you must specify column mapping to map the column from the dataset to key words that are accepted by the evaluators. In this case, we specify the data mapping for `ground_truth`.
 
 ```python
-from azure.ai.evaluation.evaluate import evaluate
+from azure.ai.evaluation import evaluate
 
 result = evaluate(
     data="data.jsonl", # provide your data here

articles/ai-studio/how-to/develop/simulator-interaction-data.md

@@ -10,9 +10,9 @@ ms.custom:
   - references_regions
 ms.topic: how-to
 ms.date: 9/24/2024
-ms.reviewer: eur
-ms.author: eur
-author: eric-urban
+ms.reviewer: minthigpen
+ms.author: lagayhar
+author: lgayhardt
 ---
 
 # Generate synthetic and simulated data for evaluation
@@ -306,8 +306,8 @@ The `AdversarialSimulator` supports a range of scenarios, hosted in the service,
 | Text Rewrite                  | `ADVERSARIAL_REWRITE`                |1000 |Hateful and unfair content, Sexual content, Violent content, Self-harm-related content, Direct Attack (UPIA) Jailbreak |
 | Ungrounded Content Generation | `ADVERSARIAL_CONTENT_GEN_UNGROUNDED` |496 | Groundedness |
 | Grounded Content Generation   | `ADVERSARIAL_CONTENT_GEN_GROUNDED`   |475 |Groundedness |
-| Protected Material | `ADVERSARIAL_PROTECTED_MATERIAL` | 200 | Protected Material |
-|Indirect Attack (XPIA) Jailbreak | `ADVERSARIAL_INDIRECT_JAILBREAK` | 200 | Indirect Attack (XPIA) Jailbreak|
+| Protected Material | `ADVERSARIAL_PROTECTED_MATERIAL` | 306 | Protected Material |
+|Indirect Attack (XPIA) Jailbreak | `ADVERSARIAL_INDIRECT_JAILBREAK` | 100 | Indirect Attack (XPIA) Jailbreak|
 
 ### Simulating jailbreak attacks
 

articles/ai-studio/toc.yml

@@ -269,7 +269,7 @@ items:
         href: concepts/evaluation-improvement-strategies.md
     - name: Manually evaluate prompts in Azure AI Studio playground
       href: how-to/evaluate-prompts-playground.md
-    - name: Generate adversarial simulations for safety evaluation
+    - name: Generate synthetic and simulated data for evaluation
       href: how-to/develop/simulator-interaction-data.md
     - name: Evaluate with the Azure AI Evaluation SDK
       href: how-to/develop/evaluate-sdk.md

articles/machine-learning/how-to-create-image-labeling-projects.md

@@ -8,7 +8,7 @@ ms.reviewer: vkann
 ms.service: azure-machine-learning
 ms.subservice: mldata
 ms.topic: how-to
-ms.date: 02/01/2024
+ms.date: 09/24/2024
 ms.custom: data4ml
 monikerRange: 'azureml-api-1 || azureml-api-2'
 # Customer intent: As a project manager, I want to set up a project to label images in the project. I want to enable machine learning-assisted labeling to help with the task.
@@ -28,7 +28,7 @@ You can also use the data labeling tool in Azure Machine Learning to [create a t
 
 Azure Machine Learning data labeling is a tool you can use to create, manage, and monitor data labeling projects. Use it to:
 
-- Coordinate data, labels, and team members to efficiently manage labeling tasks.
+- Coordinate data, labels, and team members to efficiently manage the labeling tasks.
 - Track progress and maintain the queue of incomplete labeling tasks.
 - Start and stop the project, and control the labeling progress.
 - Review and export the labeled data as an Azure Machine Learning dataset.
@@ -38,20 +38,20 @@ Azure Machine Learning data labeling is a tool you can use to create, manage, an
 
 Image data can be any file that has one of these file extensions:
 
-- *.jpg*
-- *.jpeg*
-- *.png*
-- *.jpe*
-- *.jfif*
-- *.bmp*
-- *.tif*
-- *.tiff*
-- *.dcm*
-- *.dicom*
+- `.jpg`
+- `.jpeg`
+- `.png`
+- `.jpe`
+- `.jfif`
+- `.bmp`
+- `.tif`
+- `.tiff`
+- `.dcm`
+- `.dicom`
 
 Each file is an item to be labeled.
 
-You can also use an MLTable data asset as input to an image labeling project, as long as the images in the table are one of the above formats. For more information, see [How to use MLTable data assets](./how-to-mltable.md).
+You can also use an `MLTable` data asset as input to an image labeling project, as long as the images in the table are one of the above formats. For more information, see [How to use `MLTable` data assets](./how-to-mltable.md).
 
 ## Prerequisites
 
@@ -76,10 +76,10 @@ You use these items to set up image labeling in Azure Machine Learning:
     * To apply only a *single label* to an image from a set of labels, select **Image Classification Multi-class**.
     * To apply *one or more* labels to an image from a set of labels, select **Image Classification Multi-label**. For example, a photo of a dog might be labeled with both *dog* and *daytime*.
     * To assign a label to each object within an image and add bounding boxes, select **Object Identification (Bounding Box)**.
-    * To assign a label to each object within an image and draw a polygon around each object, select **Instance Segmentation (Polygon)**.
+    * To assign a label to each object within an image and draw a polygon around each object, select **Polygon (Instance Segmentation)**.
     * To draw masks on an image and assign a label class at the pixel level, select **Semantic Segmentation (Preview)**.
 
-    :::image type="content" source="media/how-to-create-labeling-projects/labeling-creation-wizard.png" alt-text="Screenshot that shows creating a labeling project to manage labeling.":::
+    :::image type="content" source="media/how-to-create-labeling-projects/labeling-creation-wizard.png" alt-text="Screenshot that shows creating a labeling project to manage the labeling task.":::
 
 1. Select **Next** to continue.
 
@@ -98,7 +98,7 @@ You can also select **Create a dataset** to use an existing Azure datastore or t
 
 ### Data column mapping (preview)
 
-If you select an MLTable data asset, an additional **Data Column Mapping** step appears for you to specify the column that contains the image URLs.  
+If you select an MLTable data asset, another **Data Column Mapping** step appears for you to specify the column that contains the image URLs.  
 
 [!INCLUDE [mapping](includes/machine-learning-data-labeling-mapping.md)]
 
@@ -169,7 +169,7 @@ For bounding boxes, important questions include:
 * How should labelers handle an object that isn't the object class of interest but has visual similarities to a relevant object type?
 
 > [!NOTE]
-> Labelers can select the first nine labels by using number keys 1 through 9.
+> Labelers can select the first nine labels by using number keys 1 through 9. You might want to include this information in your instructions.
 
 ## Quality control (preview)
 
@@ -180,7 +180,7 @@ For bounding boxes, important questions include:
 
 ## Use ML-assisted data labeling
 
-To accelerate labeling tasks, on the **ML assisted labeling** page, you can trigger automatic machine learning models. Medical images (files that have a *.dcm* extension) aren't included in assisted labeling.  If the project type is **Semantic Segmentation (Preview)**, ML-assisted labeling isn't available.
+To accelerate labeling tasks, on the **ML assisted labeling** page, you can trigger automatic machine learning models. Medical images (files that have a `.dcm` extension) aren't included in assisted labeling. If the project type is **Semantic Segmentation (Preview)**, ML-assisted labeling isn't available.
 
 At the start of your labeling project, the items are shuffled into a random order to reduce potential bias. However, the trained model reflects any biases that are present in the dataset. For example, if 80 percent of your items are of a single class, then approximately 80 percent of the data used to train the model lands in that class.
 
@@ -189,9 +189,9 @@ To enable assisted labeling, select **Enable ML assisted labeling** and specify
 ML-assisted labeling consists of two phases:
 
 * Clustering
-* Pre-labeling
+* Prelabeling
 
-The labeled data item count that's required to start assisted labeling isn't a fixed number. This number can vary significantly from one labeling project to another. For some projects, it's sometimes possible to see pre-label or cluster tasks after 300 items have been manually labeled. ML-assisted labeling uses a technique called *transfer learning*. Transfer learning uses a pre-trained model to jump-start the training process. If the classes of your dataset resemble the classes in the pre-trained model, pre-labels might become available after only a few hundred manually labeled items. If your dataset significantly differs from the data that's used to pre-train the model, the process might take more time.
+The labeled data item count that's needed to start assisted labeling isn't a fixed number. This number can vary significantly from one labeling project to another. For some projects, it's sometimes possible to see prelabel or cluster tasks after 300 items are manually labeled. ML-assisted labeling uses a technique called *transfer learning*. Transfer learning uses a pretrained model to jump-start the training process. If the classes of your dataset resemble the classes in the pretrained model, prelabels might become available after only a few hundred manually labeled items. If your dataset significantly differs from the data that's used to pretrain the model, the process might take more time.
 
 When you use consensus labeling, the consensus label is used for training.
 
@@ -208,11 +208,11 @@ After a machine learning model is trained on your manually labeled data, the mod
 
 The clustering phase doesn't appear for object detection models or text classification.
 
-### Pre-labeling
+### Prelabeling
 
-After you submit enough labels for training, either a classification model predicts tags or an object detection model predicts bounding boxes. The labeler now sees pages that contain predicted labels already present on each item. For object detection, predicted boxes are also shown. The task involves reviewing these predictions and correcting any incorrectly labeled images before page submission.
+After you submit enough labels for training, either a classification model predicts tags, or an object detection model predicts bounding boxes. The labeler now sees pages that contain predicted labels already present on each item. For object detection, predicted boxes are also shown. The task involves reviewing these predictions and correcting any incorrectly labeled images before page submission.
 
-After a machine learning model is trained on your manually labeled data, the model is evaluated on a test set of manually labeled items. The evaluation helps determine the model's accuracy at different confidence thresholds. The evaluation process sets a confidence threshold beyond which the model is accurate enough to show pre-labels. The model is then evaluated against unlabeled data. Items with predictions that are more confident than the threshold are used for pre-labeling.
+After a machine learning model is trained on your manually labeled data, the model is evaluated on a test set of manually labeled items. The evaluation helps determine the model's accuracy at different confidence thresholds. The evaluation process sets a confidence threshold beyond which the model is accurate enough to show prelabels. The model is then evaluated against unlabeled data. Items with predictions that are more confident than the threshold are used for prelabeling.
 
 ## Initialize the image labeling project
 
@@ -224,8 +224,7 @@ After a machine learning model is trained on your manually labeled data, the mod
 [!INCLUDE [troubleshoot](includes/machine-learning-data-labeling-troubleshoot.md)]
 
 
-## Next steps
+## Related content
 
-<!-- * [Tutorial: Create your first image classification labeling project](tutorial-labeling.md). -->
 * [Manage labeling projects](how-to-manage-labeling-projects.md)
 * [How to tag images](how-to-label-data.md)

articles/machine-learning/includes/machine-learning-data-labeling-initialize.md

@@ -2,11 +2,11 @@
 author: sgilley
 ms.service: azure-machine-learning
 ms.topic: include
-ms.date: 10/21/2021
+ms.date: 09/24/2024
 ms.author: sdgilley
 ---
 
 After the labeling project is initialized, some aspects of the  project are immutable. You can't change the task type or dataset. You *can* modify labels and the URL for the task description. Carefully review the settings before you create the project. After you submit the project, you return to the **Data Labeling** overview page, which shows the project as **Initializing**.
 
 > [!NOTE]
-> This page might not automatically refresh. After a pause, manually refresh the page to see the project's status as **Created**.
+> The overview page might not automatically refresh. After a pause, manually refresh the page to see the project's status as **Created**.

articles/machine-learning/includes/machine-learning-data-labeling-refresh.md

@@ -2,7 +2,7 @@
 author: sgilley
 ms.service: azure-machine-learning
 ms.topic: include
-ms.date: 12/08/2021
+ms.date: 09/24/2024
 ms.author: sdgilley
 ---
 
@@ -15,7 +15,7 @@ Select **Enable incremental refresh at regular intervals** when you want your pr
 Clear the selection if you don't want new files in the datastore to automatically be added to your project.
 
 > [!IMPORTANT]
-> Don't create a new version for the dataset you want to update. If you do, the updates won't be seen because the data labeling project is pinned to the initial version. Instead, use [Azure Storage Explorer](https://azure.microsoft.com/features/storage-explorer/) to modify your data in the appropriate folder in Blob Storage.
+> When incremental refresh is enabled, don't create a new version for the dataset you want to update. If you do, the updates won't be seen because the data labeling project is pinned to the initial version. Instead, use [Azure Storage Explorer](https://azure.microsoft.com/features/storage-explorer/) to modify your data in the appropriate folder in Blob Storage.
 >
 > Also, don't remove data. Removing data from the dataset your project uses causes an error in the project. 
 

articles/search/hybrid-search-how-to-query.md

@@ -28,7 +28,7 @@ To improve relevance, use these parameters:
 
 + A search index containing `searchable` vector and nonvector fields. See [Create an index](search-how-to-create-search-index.md) and [Add vector fields to a search index](vector-search-how-to-create-index.md).
 
-+ (Optional) If you want [semantic ranking](semantic-how-to-configure.md), your search service must be Basic tier or higher, with [semantic ranking enabled](semantic-how-to-enable-disable.md).
++ (Optional) If you want the [semantic ranker](semantic-search-overview.md), your search service must be Basic tier or higher, with [semantic ranker enabled](semantic-how-to-enable-disable.md).
 
 + (Optional) If you want text-to-vector conversion of a query string, [create and assign a vectorizer](vector-search-how-to-configure-vectorizer.md) to vector fields in the search index.
 
@@ -167,7 +167,7 @@ api-key: {{admin-api-key}}
 
 ## Semantic hybrid search
 
-Assuming that you [enabled semantic ranking](semantic-how-to-enable-disable.md) and your index definition includes a [semantic configuration](semantic-how-to-query-request.md), you can formulate a query that includes vector search and keyword search, with semantic ranking over the merged result set. Optionally, you can add captions and answers. 
+Assuming that you [enabled semantic ranker](semantic-how-to-enable-disable.md) and your index definition includes a [semantic configuration](semantic-how-to-query-request.md), you can formulate a query that includes vector search and keyword search, with semantic ranking over the merged result set. Optionally, you can add captions and answers. 
 
 ```http
 POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2024-07-01
@@ -200,7 +200,7 @@ api-key: {{admin-api-key}}
 
 **Key points:**
 
-+ Semantic ranking accepts up to 50 results from the merged response.
++ Semantic ranker accepts up to 50 results from the merged response.
 
 + "queryType" and "semanticConfiguration" are required.
 
@@ -353,9 +353,9 @@ Both "k" and "top" are optional. Unspecified, the default number of results in a
 > [!NOTE]
 > The semantic ranker can take up to 50 results. 
 
-If you're using semantic ranking in 2024-05-01-preview API, it's a best practice to set "k" and "maxTextRecallSize" to sum to at least 50 total.  You can then restrict the results returned to the user with the "top" parameter. 
+If you're using semantic ranker in 2024-05-01-preview API, it's a best practice to set "k" and "maxTextRecallSize" to sum to at least 50 total.  You can then restrict the results returned to the user with the "top" parameter. 
 
-If you're using semantic ranking in previous APIs do the following:
+If you're using semantic ranker in previous APIs do the following:
 
 + if doing keyword-only search (no vector) set "top" to 50
 + if doing hybrid search set "k" to 50, to ensure that the semantic ranker gets at least 50 results.