Merge pull request #275313 from MicrosoftDocs/release-build-2024-appliedai-language

Release build 2024 appliedai language

Author: WendyRing

articles/ai-services/language-service/personally-identifiable-information/concepts/conversations-entity-categories.md

@@ -1,7 +1,7 @@
 ---
 title: Entity categories recognized by Conversational Personally Identifiable Information (detection) in Azure AI Language
 titleSuffix: Azure AI services
-description: Learn about the entities the Conversational PII feature (preview) can recognize from conversation inputs.
+description: Learn about the entities the Conversational PII feature can recognize from conversation inputs.
 #services: cognitive-services
 author: jboback
 manager: nitinme

articles/ai-services/language-service/personally-identifiable-information/language-support.md

@@ -14,7 +14,7 @@ ms.custom: language-service-pii
 
 # Personally Identifiable Information (PII) detection language support 
 
-Use this article to learn which natural languages are supported by the PII and conversation PII (preview) features of Azure AI Language.
+Use this article to learn which natural languages are supported by the PII and conversation PII features of Azure AI Language.
 
 # [PII for documents](#tab/documents)
 

articles/ai-services/language-service/personally-identifiable-information/overview.md

@@ -14,17 +14,15 @@ ms.custom: language-service-pii
 
 # What is Personally Identifiable Information (PII) detection in Azure AI Language?
 
-PII detection is one of the features offered by [Azure AI Language](../overview.md), a collection of machine learning and AI algorithms in the cloud for developing intelligent applications that involve written language. The PII detection feature can **identify, categorize, and redact** sensitive information in unstructured text. For example: phone numbers, email addresses, and forms of identification. The method for utilizing PII in conversations is different than other use cases, and articles for this use are separate.
+As of June 2024, we now provide General Availability support for the Conversational PII service (English-language only).
+Customers can now redact transcripts, chats, and other text written in a conversational style (i.e. text with “um”s, “ah”s, multiple speakers, and the spelling out of words for more clarity) with better confidence in AI quality, Azure SLA support and production environment support, and enterprise-grade security in mind.
+
+PII detection is one of the features offered by [Azure AI Language](../overview.md), a collection of machine learning and AI algorithms in the cloud for developing intelligent applications that involve written language. The PII detection feature can **identify, categorize, and redact** sensitive information in unstructured text. For example: phone numbers, email addresses, and forms of identification. Azure AI Language supports general text PII redaction, as well as [Conversational PII](how-to-call-for-conversations.md), a specialized model for handling speech transcriptions and the more informal, conversational tone of meeting and call transcripts. The service also supports [Native Document PII redaction](#native-document-support), where the input and output are structured document files.
 
 * [**Quickstarts**](quickstart.md) are getting-started instructions to guide you through making requests to the service.
 * [**How-to guides**](how-to-call.md) contain instructions for using the service in more specific or customized ways.
 * The [**conceptual articles**](concepts/entity-categories.md) provide in-depth explanations of the service's functionality and features.
 
-PII comes into two shapes:
-
-* [PII](how-to-call.md) - works on unstructured text.
-* [Conversation PII (preview)](how-to-call-for-conversations.md) - tailored model to work on conversation transcription.
-
 [!INCLUDE [Typical workflow for pre-configured language features](../includes/overview-typical-workflow.md)]
 
 ## Native document support

articles/ai-services/language-service/summarization/custom/how-to/data-formats.md

@@ -21,7 +21,7 @@ This page contains information about how to select and prepare data in order to
 
 ## Custom summarization document sample format
 
-In the abstractive document summarization scenario, each document (whether it has a provided label or not) is expected to be provided in a plain .txt file. The file contains one or more lines. If multiple lines are provided, each is assumed to be a paragraph of the document. The following is an example document with three paragraphs.
+In the abstractive text summarization scenario, each document (whether it has a provided label or not) is expected to be provided in a plain .txt file. The file contains one or more lines. If multiple lines are provided, each is assumed to be a paragraph of the document. The following is an example document with three paragraphs.
 
 *At Microsoft, we have been on a quest to advance AI beyond existing techniques, by taking a more holistic, human-centric approach to learning and understanding. As Chief Technology Officer of Azure AI services, I have been working with a team of amazing scientists and engineers to turn this quest into a reality.*
 
@@ -66,7 +66,7 @@ In the abstractive document summarization scenario, each document (whether it ha
 
 ## Sample mapping JSON format
 
-In both document and conversation summarization scenarios, a set of documents and corresponding labels can be provided in a single JSON file that references individual document/conversation and summary files. 
+In both text and conversation summarization scenarios, a set of documents and corresponding labels can be provided in a single JSON file that references individual document/conversation and summary files. 
 
 The JSON file is expected to contain the following fields:
 
@@ -96,7 +96,7 @@ The JSON file is expected to contain the following fields:
 ```
 ## Custom document summarization mapping sample
 
-The following is an example mapping file for the abstractive document summarization scenario with three documents and corresponding labels.
+The following is an example mapping file for the abstractive text summarization scenario with three documents and corresponding labels.
 
 ```json
 {

articles/ai-services/language-service/summarization/how-to/conversation-summarization.md

@@ -47,8 +47,8 @@ For easier navigation, here are links to the corresponding sections for each ser
 
 The conversation summarization API uses natural language processing techniques to summarize conversations into shorter summaries per request. Conversation summarization can summarize for issues and resolutions discussed in a two-party conversation or summarize a long conversation into chapters and a short narrative for each chapter.
 
-There's another feature in Azure AI Language named [document summarization](../overview.md?tabs=document-summarization) that is more suitable to summarize documents into concise summaries. When you're deciding between document summarization and conversation summarization, consider the following points:
-* Input format: Conversation summarization can operate on both chat text and speech transcripts, which have speakers and their utterances. Document summarization operates using simple text, or Word, PDF, or PowerPoint formats.
+There's another feature in Azure AI Language named [text summarization](../overview.md?tabs=text-summarization) that is more suitable to summarize documents into concise summaries. When you're deciding between text summarization and conversation summarization, consider the following points:
+* Input format: Conversation summarization can operate on both chat text and speech transcripts, which have speakers and their utterances. Text summarization operates using simple text, or Word, PDF, or PowerPoint formats.
 * Purpose of summarization: for example, conversation issue and resolution summarization returns a reason and the resolution for a chat between a customer and a customer service agent.
 
 ## Submitting data

articles/ai-services/language-service/summarization/how-to/document-summarization.md

@@ -14,9 +14,9 @@ ms.custom:
   - ignite-2023
 ---
 
-# How to use document summarization
+# How to use text summarization
 
-Document summarization is designed to shorten content that users consider too long to read. Both extractive and abstractive summarization condense articles, papers, or documents to key sentences.
+Text summarization is designed to shorten content that users consider too long to read. Both extractive and abstractive summarization condense articles, papers, or documents to key sentences.
 
 **Extractive summarization**: Produces a summary by extracting sentences that collectively represent the most important or relevant information within the original content.
 
@@ -32,8 +32,8 @@ For easier navigation, here are links to the corresponding sections for each ser
 
 |Aspect       |Section                                                            |
 |-------------|-------------------------------------------------------------------|
-|Extractive   |[Extractive Summarization](#try-document-extractive-summarization) |
-|Abstractive  |[Abstrctive Summarization](#try-document-abstractive-summarization)|
+|Extractive   |[Extractive Summarization](#try-text-extractive-summarization) |
+|Abstractive  |[Abstractive Summarization](#try-text-abstractive-summarization)|
 |Query-focused|[Query-focused Summarization](#query-based-summarization)          |
 
 
@@ -61,15 +61,15 @@ You submit documents to the API as strings of text. Analysis is performed upon r
 
 When you use this feature, the API results are available for 24 hours from the time the request was ingested, and is indicated in the response. After this time period, the results are purged and are no longer available for retrieval.
 
-### Getting document summarization results
+### Getting text summarization results
 
 When you get results from language detection, you can stream the results to an application or save the output to a file on the local system.
 
 The following is an example of content you might submit for summarization, which is extracted using the Microsoft blog article [A holistic representation toward integrative AI](https://www.microsoft.com/research/blog/a-holistic-representation-toward-integrative-ai/). This article is only an example, the API can accept longer input text. See the data limits section for more information.
 
 *"At Microsoft, we have been on a quest to advance AI beyond existing techniques, by taking a more holistic, human-centric approach to learning and understanding. As Chief Technology Officer of Azure AI services, I have been working with a team of amazing scientists and engineers to turn this quest into a reality. In my role, I enjoy a unique perspective in viewing the relationship among three attributes of human cognition: monolingual text (X), audio or visual sensory signals, (Y) and multilingual (Z). At the intersection of all three, there’s magic—what we call XYZ-code as illustrated in Figure 1—a joint representation to create more powerful AI that can speak, hear, see, and understand humans better. We believe XYZ-code enables us to fulfill our long-term vision: cross-domain transfer learning, spanning modalities and languages. The goal is to have pretrained models that can jointly learn representations to support a broad range of downstream AI tasks, much in the way humans do today. Over the past five years, we have achieved human performance on benchmarks in conversational speech recognition, machine translation, conversational question answering, machine reading comprehension, and image captioning. These five breakthroughs provided us with strong signals toward our more ambitious aspiration to produce a leap in AI capabilities, achieving multi-sensory and multilingual learning that is closer in line with how humans learn and understand. I believe the joint XYZ-code is a foundational component of this aspiration, if grounded with external knowledge sources in the downstream AI tasks."*
 
-The document summarization API request is processed upon receipt of the request by creating a job for the API backend. If the job succeeded, the output of the API is returned. The output is available for retrieval for 24 hours. After this time, the output is purged. Due to multilingual and emoji support, the response might contain text offsets. See [how to process offsets](../../concepts/multilingual-emoji-support.md) for more information.
+The text summarization API request is processed upon receipt of the request by creating a job for the API backend. If the job succeeded, the output of the API is returned. The output is available for retrieval for 24 hours. After this time, the output is purged. Due to multilingual and emoji support, the response might contain text offsets. See [how to process offsets](../../concepts/multilingual-emoji-support.md) for more information.
 
 When you use the above example, the API might return the following summarized sentences:
 
@@ -81,9 +81,9 @@ When you use the above example, the API might return the following summarized se
 **Abstractive summarization**:
 - "Microsoft is taking a more holistic, human-centric approach to learning and understanding. We believe XYZ-code enables us to fulfill our long-term vision: cross-domain transfer learning, spanning modalities and languages. Over the past five years, we have achieved human performance on benchmarks in."
 
-### Try document extractive summarization
+### Try text extractive summarization
 
-You can use document extractive summarization to get summaries of articles, papers, or documents. To see an example, see the [quickstart article](../quickstart.md).
+You can use text extractive summarization to get summaries of articles, papers, or documents. To see an example, see the [quickstart article](../quickstart.md).
 
 You can use the `sentenceCount` parameter to guide how many sentences are returned, with `3` being the default. The range is from 1 to 20.
 
@@ -94,20 +94,20 @@ You can also use the `sortby` parameter to specify in what order the extracted s
 |Rank    | Order sentences according to their relevance to the input document, as decided by the service.        |
 |Offset    | Keeps the original order in which the sentences appear in the input document.        |
 
-### Try document abstractive summarization
+### Try text abstractive summarization
 
-The following example gets you started with document abstractive summarization:
+The following example gets you started with text abstractive summarization:
 
 1. Copy the command below into a text editor. The BASH example uses the `\` line continuation character. If your console or terminal uses a different line continuation character, use that character instead.
 
 ```bash
-curl -i -X POST https://<your-language-resource-endpoint>/language/analyze-text/jobs?api-version=2022-10-01-preview \
+curl -i -X POST https://<your-language-resource-endpoint>/language/analyze-text/jobs?api-version=2023-04-01 \
 -H "Content-Type: application/json" \
 -H "Ocp-Apim-Subscription-Key: <your-language-resource-key>" \
 -d \
 ' 
 {
-  "displayName": "Document Abstractive Summarization Task Example",
+  "displayName": "Text Abstractive Summarization Task Example",
   "analysisInput": {
     "documents": [
       {
@@ -120,7 +120,7 @@ curl -i -X POST https://<your-language-resource-endpoint>/language/analyze-text/
   "tasks": [
     {
       "kind": "AbstractiveSummarization",
-      "taskName": "Document Abstractive Summarization Task 1",
+      "taskName": "Text Abstractive Summarization Task 1",
       "parameters": {
         "summaryLength": short
       }
@@ -153,7 +153,7 @@ curl -X GET https://<your-language-resource-endpoint>/language/analyze-text/jobs
 -H "Ocp-Apim-Subscription-Key: <your-language-resource-key>"
 ```
 
-### Abstractive document summarization example JSON response
+### Abstractive text summarization example JSON response
 
 ```json
 {
@@ -163,7 +163,7 @@ curl -X GET https://<your-language-resource-endpoint>/language/analyze-text/jobs
     "expirationDateTime": "2022-09-09T16:44:53Z",
     "status": "succeeded",
     "errors": [],
-    "displayName": "Document Abstractive Summarization Task Example",
+    "displayName": "Text Abstractive Summarization Task Example",
     "tasks": {
         "completed": 1,
         "failed": 0,
@@ -172,7 +172,7 @@ curl -X GET https://<your-language-resource-endpoint>/language/analyze-text/jobs
         "items": [
             {
                 "kind": "AbstractiveSummarizationLROResults",
-                "taskName": "Document Abstractive Summarization Task 1",
+                "taskName": "Text Abstractive Summarization Task 1",
                 "lastUpdateDateTime": "2022-09-08T16:45:14.0717206Z",
                 "status": "succeeded",
                 "results": {
@@ -212,7 +212,7 @@ The following cURL commands are executed from a BASH shell. Edit these commands
 
 ## Query based summarization
 
-The query-based document summarization API is an extension to the existing document summarization API.
+The query-based text summarization API is an extension to the existing text summarization API.
 
 The biggest difference is a new `query` field in the request body (under `tasks` > `parameters` > `query`). Additionally, there's a new way to specify the preferred `summaryLength` in "buckets" of short/medium/long, which we recommend using instead of `sentenceCount`, especially when using abstractive. Below is an example request:
 
@@ -223,7 +223,7 @@ curl -i -X POST https://<your-language-resource-endpoint>/language/analyze-text/
 -d \
 ' 
 {
-  "displayName": "Document Extractive Summarization Task Example",
+  "displayName": "Text Extractive Summarization Task Example",
   "analysisInput": {
     "documents": [
       {
@@ -233,13 +233,20 @@ curl -i -X POST https://<your-language-resource-endpoint>/language/analyze-text/
       }
     ]
   },
-  "tasks": [
+"tasks": [
     {
+      "kind": "AbstractiveSummarization",
+      "taskName": "Query-based Abstractive Summarization",
+      "parameters": {
+          "query": "XYZ-code",
+          "summaryLength": "short"
+      }
+    },    {
       "kind": "ExtractiveSummarization",
-      "taskName": "Document Extractive Summarization Task 1",
+      "taskName": "Query_based Extractive Summarization",
       "parameters": {
-        "query": "XYZ-code",
-        "summaryLength": short
+          "query": "XYZ-code",
+          "sentenceCount": 3
       }
     }
   ]
@@ -249,6 +256,7 @@ curl -i -X POST https://<your-language-resource-endpoint>/language/analyze-text/
 
 ### Using the summaryParameter
 For the `summaryLength` parameter, three values are accepted:
+* oneSentence: Generates a summary of mostly 1 sentence, with around 80 tokens.
 * short: Generates a summary of mostly 2-3 sentences, with around 120 tokens.
 * medium: Generates a summary of mostly 4-6 sentences, with around 170 tokens.
 * long: Generates a summary of mostly over 7 sentences, with around 210 tokens.