Merge pull request #4824 from laujan/mohamed-4701-custom-final

Mohamed 4701 custom final

Author: ShannonLeavitt

articles/ai-services/translator/custom-translator/azure-ai-foundry/beginners-guide.md

@@ -139,11 +139,11 @@ To prepare for training, documents undergo a series of processing and filtering
 
 * ### Invalid characters
 
-  custom translation removes sentences that contain Unicode character U+FFFD. The character U+FFFD indicates a failed encoding conversion.
+  Custom translation removes sentences that contain Unicode character U+FFFD. The character U+FFFD indicates a failed encoding conversion.
 
 * ### Invalid HTML tags
 
-  custom translation removes valid tags during training. Invalid tags cause unpredictable results and should be manually removed. 
+  Custom translation removes valid tags during training. Invalid tags cause unpredictable results and should be manually removed. 
 
 ## What steps should I take before uploading data?
 

articles/ai-services/translator/custom-translator/azure-ai-foundry/how-to/create-language-pair.md

@@ -16,7 +16,9 @@ An Azure AI Foundry custom translation language pair includes models, training,
 
 ## Create a language pair
 
-1. Follow [Create a project](create-language-pair.md), then continue here.
+1. Follow [Create a project](create-project.md), then continue here.
+
+1. Use the dropdown list to select another **Connected service** or create a new one.
 
 1. Enter the following details about your language pair in the dialog:
 
@@ -33,13 +35,13 @@ An Azure AI Foundry custom translation language pair includes models, training,
 
    - **Language pair label:** You can add label to create the same language pair multiple times. Example, you want to create English to French model for shopping and another English to French model for automotive. A label distinguishes between the same language pair with the same language pair and domain. As a best practice, here are a few tips:
 
-   - Use a label *only* if you're planning to build multiple projects for the same language pair and same domain and want to access these projects with a different Domain ID.
+      - Use a label *only* if you're planning to build multiple projects for the same language pair and same domain and want to access these projects with a different Domain ID.
 
-   - Don't use a label if you're building systems for one domain only.
+      - Don't use a label if you're building systems for one domain only.
 
-   - A label isn't required and not helpful to distinguish between language pairs.
+      - A label isn't required and not helpful to distinguish between language pairs.
 
-   - You can use the same label for multiple language pairs.
+      - You can use the same label for multiple language pairs.
 
    - **Project description:** A short summary about the project. This description has no influence over the behavior of the Custom Translator or your resulting custom system, but can help you differentiate between different projects.
 
@@ -55,10 +57,9 @@ To modify the language pair name, description, domain description, and add/remov
 
 1. Select the language pair name from the Fine-tuning > AI Service fine-tuning page.
 
-1. Select the **...** next to the language pair name and select **Edit**.
+   :::image type="content" source="../media/fine-tune-edit-language-pair-1.png" alt-text="Screenshot illustrating edit language pair fields.":::
 
-1. The **Edit and Delete** buttons should now be visible.
-   :::image type="content" source="../media/fine-tune-edit-language-pair-1.png" alt-text="Screenshot illustrating edit language pair fields":::
+1. Select the **`...`** next to the language pair name. The **Edit and Delete** buttons should now be visible.
 
 1. Select **Edit** and fill in or modify existing text.
 

articles/ai-services/translator/custom-translator/azure-ai-foundry/how-to/test-model.md

@@ -33,7 +33,7 @@ A BLEU score is a number between zero and 100. A score of zero indicates a low-q
 
    :::image type="content" source="../media/fine-tune-test-model-2.png" alt-text="Screenshot illustrating the test-model function.":::
 
-1. If results are satisfactory, deploy the model to production, otherwise, iterate by adding more human curated training data until you find a winner model.
+1. If results are satisfactory, deploy the model, otherwise, iterate by adding more human curated training data until you find a winner model.
 
 ## Next steps
 

articles/ai-services/translator/custom-translator/azure-ai-foundry/how-to/train-model.md

@@ -35,11 +35,13 @@ Here are the steps to create a custom translation model:
 
    :::image type="content" source="../media/fine-tune-train-model-2.png" alt-text="Screenshot illustrating train model blade.":::
 
-1. Select the data you want to use for training, for example, `Customer-sample-English-German-Training` and review the training cost associated with the selected number of sentences.
+1. Select the data you want to use for training and review the training cost associated with the selected number of sentences.
 
    :::image type="content" source="../media/fine-tune-train-model-3.png" alt-text="Screenshot depicting a view of the train model blade.":::
 
-1. Select **Train model**.
+1. Select **Next**
+
+1. Review and select **Train model**.
 
    :::image type="content" source="../media/fine-tune-train-model-4.png" alt-text="Screenshot illustrating the train model blade.":::
 

articles/ai-services/translator/custom-translator/azure-ai-foundry/how-to/upload-data.md

@@ -14,21 +14,13 @@ ms.topic: how-to
 
 Document types are associated with the language pair selected when you create a project.
 
-1. Sign-in to [Azure AI Foundry](https://ai.azure.com/). 
-
-1. Select **Fine-tuning** from the left pane.
+1. Follow [Create a language pair](create-language-pair.md), then continue here. 
 
 1. Select the language pair **Name** you want to manage.
 
-1. Select **Manage data** and then select **Add data** and choose the dataset type:
-
-    - Training set
-    - Testing set
-    - Tuning set
-    - Phrase Dictionary
-    - Sentence Dictionary
+   :::image type="content" source="../media/fine-tune-edit-language-pair-1.png" alt-text="Screenshot illustrating edit language pair fields.":::
 
-   :::image type="content" source="../media/fine-tune-create-language-pair.png" alt-text="Screenshot illustrating the document upload link.":::
+1. Select **Manage data** and then select **Add data**. 
 
 1. Select dataset type and select your data document format.
 
@@ -46,7 +38,7 @@ You can view history of all document uploads details like document type, upload
 
 1. The upload history tab shows history for the selected language pair.
 
-2. This page shows the status of all of your past uploads. It displays
+1. This page shows the status of all of your past uploads. It displays
     uploads from most recent to least recent. Each upload shows name, status, type, and created on.
 
    :::image type="content" source="../media/fine-tune-upload-history.png" alt-text="Screenshot showing the upload history page.":::

articles/ai-services/translator/custom-translator/azure-ai-foundry/overview.md

@@ -34,7 +34,7 @@ Custom translation provides different features to build custom translation syste
 
 ## Get better translations
 
-Azure AI Translator released [Neural Machine Translation (NMT)](https://www.microsoft.com/translator/blog/2016/11/15/microsoft-translator-launching-neural-network-based-translations-for-all-its-speech-languages/) in 2016. NMT provided major advances in translation quality over the industry-standard [Statistical Machine Translation (SMT)](https://en.wikipedia.org/wiki/Statistical_machine_translation) technology. Because NMT better captures the context of full sentences before translating them, it provides higher quality, more human-sounding, and more fluent translations. [Custom Translator](https://ai.azure.com) provides NMT for your custom models resulting in better translation quality.
+Azure AI Translator released [Neural Machine Translation (NMT)](https://www.microsoft.com/translator/blog/2016/11/15/microsoft-translator-launching-neural-network-based-translations-for-all-its-speech-languages/) in 2016. NMT provided major advances in translation quality over the industry-standard [Statistical Machine Translation (SMT)](https://en.wikipedia.org/wiki/Statistical_machine_translation) technology. Because NMT better captures the context of full sentences before translating them, it provides higher quality, more human-sounding, and more fluent translations. [Custom translation](https://ai.azure.com) provides NMT for your custom models resulting in better translation quality.
 
 Custom translation also accepts data that's parallel at the document level to make data collection and preparation more effective. If users have access to versions of the same content in multiple languages but in separate documents, custom translation is able to automatically match sentences across documents. For a list of supported document format, *see* [Custom translation document formats and naming convention](concepts/document-formats-naming-convention.md).
 

articles/ai-services/translator/custom-translator/release-notes.md

@@ -64,7 +64,7 @@ This page presents the latest feature, improvement, bug fix, and known issue rel
 |English (`en`)|Turkish (`tr`)|
 |English (`en`)|Ukrainian (`uk`)|
 |English (`en`)|Vietnamese (`vi`)|
-|English (`en`)|Chinese Simplified (zh-hans)|
+|English (`en`)|zh-hans|
 |Estonian (`et`)|English (`en`)|
 |Finnish (`fi`)|English (`en`)|
 |French (`fr`)|English (`en`)|

articles/ai-services/translator/text-translation/how-to/migrate-to-v4.md

@@ -1,5 +1,5 @@
 ---
-title: Migrate to v4.0 - Azure AI Translator
+title: Migrate from Translator v3 to the latest Azure AI Translator version
 titleSuffix: Azure AI services
 description: This article provides the steps to help you migrate from Azure AI Translator v3 to  2025-05-01-preview Text translation API.
 author: laujan
@@ -12,7 +12,7 @@ ms.author: lajanuar
 
 # Azure AI Translator 2025-05-01-preview migration
 
-Azure AI Translator text translation 2025-05-01-preview (v4.0) is our latest cloud-based, multilingual neural machine translation service. As Azure AI Translator matures, we're focused on patterns and practices to best support and add value to our users.
+Azure AI Translator text translation 2025-05-01-preview is our latest cloud-based, multilingual neural machine translation service. As Azure AI Translator matures, we're focused on patterns and practices to best support and add value to our users.
 
 >[!IMPORTANT]
 > * Azure AI Translator REST API `2025-05-01-preview` is new version of the Azure AI Translator REST API **with breaking changes**.
@@ -52,10 +52,10 @@ The following table compares Translator `2025-05-01-preview` and Translator v3 A
 |[Translate text](../reference/v4/translate-api.md)|[Translate text](../reference/v3/translate.md)|
 |[Transliterate](../reference/v4/transliterate-api.md)|[Transliterate](../reference/v3/transliterate.md)|
 |[Languages](../reference/v4/get-languages.md)|[Languages](../reference/v3/languages.md)|
-|Feature no longer supported|[Detect language](../reference/v3/detect.md)|
-|Feature no longer supported|[BreakSentence](../reference/v3/break-sentence.md)|
-|Feature no longer supported|[Dictionary Lookup](../reference/v3/dictionary-lookup.md)|
-|Feature no longer supported|[Dictionary Examples](../reference/v3/dictionary-examples.md)|
+[BreakSentence](../reference/v3/break-sentence.md)|Feature no longer supported.<br>Use sentence delimiters function or a Natural Language Processing (NLP) library supported for your programming language.|
+[Dictionary Lookup](../reference/v3/dictionary-lookup.md)|Feature no longer supported|
+|[Dictionary Examples](../reference/v3/dictionary-examples.md)|Feature no longer supported|
+
 
 ## Next Steps
 

articles/ai-services/translator/text-translation/reference/v4/reference-overview.md

@@ -6,7 +6,7 @@ author: laujan
 manager: nitinme
 ms.service: azure-ai-translator
 ms.topic: reference
-ms.date: 04/18/2025
+ms.date: 05/05/2025
 ms.author: lajanuar
 ---
 
@@ -16,7 +16,6 @@ Azure AI Translator `2025-05-01-preview` is our latest cloud-based, multilingual
 
 The Translator service is an optimal solution for managing extensive multilingual content. It easily integrates with your applications and workflows through a single REST API call and supports multiple programming languages. Translator supports over 100 languages and dialects, making it ideal for businesses, developers, and organizations seeking to seamlessly integrate multilingual communication.  
 
-Azure AI Translator prioritizes data security and privacy, complying with regulations like GDPR, HIPAA, and ISO/SOC, thus ensuring that it's a reliable solution for handling sensitive and confidential information.
 
 >[!IMPORTANT]
 > * Azure AI Translator REST API `2025-05-01-preview` is new version of the Azure AI Translator REST API **with breaking changes**.
@@ -28,7 +27,7 @@ Azure AI Translator prioritizes data security and privacy, complying with regula
 
 * **`LLM` choice**. You can choose a large language model for translation based on factors such as quality, cost, and other considerations.
 
-* **Adaptive custom translation**. You can provide reference translations or translation memory datasets to enable an `LLM` model to perform few-shot translations tailored to your needs. Few-shot translation is a machine translation method where the model is trained or fine-tuned with only a limited number of examples to translate between languages.
+* **Adaptive custom translation**. You can provide reference translations or translation memory datasets to enable an `LLM` model to perform few-shot translations tailored to your needs. 
 
 * The **Translator** resource doesn't support Neural Machine Translation (`NMT`) translations.
 
@@ -40,21 +39,61 @@ Metrics allow you to view the translator usage and availability information in A
 
 :::image type="content" source="../../../media/azure-portal-metrics-v4.png" alt-text="Screenshot of HTTP request metrics in the Azure portal.":::
 
-This table lists available metrics with description of how they're used to monitor translation API calls.
+#### Metrics terminology
+
+* **PTU**: provisioned throughput units
+* **TPS**: transactions per second
+* **TPM**: tokens per minute
+
+The following tables list available metrics with description of how they're used to monitor **Translator resource** API calls.
+
+#### Translator resource HTTP requests
+
+| Metrics | Description |
+|:----|:-----|
+| `BlockedCalls`| Number of calls that exceeded rate or quota limit.|
+| `ClientErrors`| Number of calls with client-side error(4XX).|
+| `Latency`| Duration to complete request in milliseconds.|
+| `Ratelimit`| The current rate limit of the rate limit key.|
+| `ServerErrors`| Number of calls with server internal error(5XX).|
+| `SuccessfulCalls`| Number of successful calls.|
+| `TotalCalls`| Total number of API calls.|
+| `TotalErrors`| Number of calls with error response.|
+| `TotalTokenCalls`| Total number of API calls via token service using authentication token.|
+
+#### Translator resource usage
+
+| Metrics | Description |
+|:----|:-----|
+| `TextCharactersTranslated`|Number of characters in incoming text translation request.|
+| `TextCustomCharactersTranslated`|Number of characters in incoming custom text translation request.|
+| `TextTrainedCharacters`|Number of characters trained using text translation.|
+| `DocumentCharactersTranslated`|Number of characters in document translation request.|
+| `DocumentCustomCharactersTranslated`|Number of characters in custom document translation request.|
+
+The following tables list available metrics with description of how they're used to monitor **Azure OpenAI** API calls.
+
+#### Azure OpenAI HTTP requests
 
 | Metrics | Description |
 |:----|:-----|
-| BlockCalls| Number of calls that exceed rate or quota.|
-| ClientErrors| Number of calls with client-side error (HTTP response code 4xx)|
-| DataIn| Size of incoming data in bytes.|
-| DataOut| Size of outgoing data in bytes.|
-| Latency| Duration to complete request in milliseconds.|
-| RateLimit| The current rate limit of the ratelimit key.|
-| ServerErrors| Number of calls with server internal error (HTTP response code 5xx).|
-| SuccessfulCalls| Number of successful calls (HTTP response code 2xx).|
-| TotalCalls| Total number of calls.|
-| TotalErrors|Total number of calls with error response (HTTP response code 4xx or 5xx)|
-| TotalTokenCalls|Total number of token calls|
+| `AzureOpenAIAvailabilityRate`|Availability percentage with the following calculation:<br>`(Total Calls - Server Errors) / Total Calls`. Server Errors include any HTTP response >= 500.|
+|`AzureOpenAIRequests`|Number of calls made to the Azure OpenAI API over a period of time. Applies to `PTU`, `PTU`-managed, and Pay-as-you-go deployments. To breakdown API requests, you can add a filter or apply splitting by the following dimensions: <br> `ModelDeploymentName`, `ModelName`, `ModelVersion`, `StatusCode` (successful, client errors, server errors), `StreamType` (streaming vs nonstreaming requests), and `Operation`.|
+
+#### Azure OpenAI usage
+
+| Metrics | Description |
+|:----|:-----|
+|`ActiveTokens`|Total tokens minus cached tokens over a period of time. Applies to `PTU` and `PTU`-managed deployments. Use this metric to understand your `TPS`- or `TPM`-based utilization for `PTU`s and compare your benchmarks for target `TPS` or `TPM` for your scenarios. <br> To breakdown API requests, you can add a filter or apply splitting by the following dimensions: `ModelDeploymentName`, `ModelName`, `ModelVersion`.|
+|`GeneratedTokens`|Number of tokens generated (output) from an OpenAI model. Applies to `PTU`, `PTU`-managed, and Pay-as-you-go deployments. To analyze this metric in detail, you can add a filter or apply splitting by the following dimensions:<br>`ModelDeploymentName`or `ModelName`.|
+|`FineTunedTrainingHours`|Number of training hours processed on an OpenAI fine-tuned model.|
+|`TokenTransaction`|Number of inference tokens processed on an OpenAI model. Calculated as prompt tokens (input) plus generated tokens (output). Applies to `PTU`, `PTU`-managed, and Pay-as-you-go deployments. To analyze this metric in detail, you can add a filter or apply splitting by the following dimensions:<br>`ModelDeploymentName`or `ModelName`.|
+|`ProcessedPromptTokens`|Number of prompt tokens processed (input) on an OpenAI model. Applies to `PTU`, `PTU`-managed, and Pay-as-you-go deployments. To analyze this metric in detail, you can add a filter or apply splitting by the following dimensions:<br>`ModelDeploymentName`or `ModelName`.|
+|`AzureOpenAIContextTokensCacheMatchRate`|Percentage of prompt tokens that hit the cache. Applies to `PTU` and `PTU`-managed deployments.
+|`AzureOpenAIProvisionedManagedUtilizationV2`|Utilization percentage for a provisioned-managed deployment, calculated as (`PTU`s consumed / `PTU`s deployed) x 100. When utilization is greater than or equal to 100%, calls are throttled and error code 429 is returned. To analyze this metric in detail, you can add a filter or apply splitting by the following dimensions: `ModelDeploymentName`, `ModelName`, `ModelVersion`, and `StreamType` (streaming vs nonstreaming requests).|
+
+
+
 
 ## Next steps