CLU 1.0 GA SDK (Azure#24649)

* update autorest file and regen

* update samples and tests and record

* update samples readme

* update changelog

* update readme

* update sdk version

* tmp update for changelog

* add convenience method

* add sample "analyze query with conv app"

* add remaining samples

* fix patch file

* add async operation

* add async samples

* remove convenience samples

* revert _patch files

* update changelog

* update version and regen

* remove unneccessary directives from autorest gen file

* add projects autorest file

* generate projects client

* update autorest readme

* regen clients

* cherry-pick swagger readme

* regen

* minor fix

* update changelog

* update dev requirements

* update shared requirements

* add authoring samples

* update readme

* add reference to azure language portal

* fix error in dev requirements arr

* update changelog

* ignore security warnings in _serialization files

Author: mshaban-msft

sdk/cognitivelanguage/azure-ai-language-conversations/CHANGELOG.md

@@ -10,6 +10,11 @@
 * Client now uses python dictionaries for method parameters and results instead of classes.
 * Many input and result parameter name changes in `analyze_conversation()` method
 
+## 1.0.0 (Unreleased)
+
+### Features Added
+* Added more resolution types for entities
+* Added support for authoring client
 
 ## 1.0.0b3 (2022-04-19)
 

sdk/cognitivelanguage/azure-ai-language-conversations/README.md

@@ -4,8 +4,7 @@
 Conversational Language Understanding - aka **CLU** for short - is a cloud-based conversational AI service which provides many language understanding capabilities like:
 - Conversation App: It's used in extracting intents and entities in conversations
 - Workflow app: Acts like an orchestrator to select the best candidate to analyze conversations to get best response from apps like Qna, Luis, and Conversation App
-- Conversational Summarization: Used to summarize conversations in the form of issues, and final resolutions
-- Conversational PII: Used to extract and redact personally-identifiable info (PII) 
+
 
 [Source code][conversationallanguage_client_src] | [Package (PyPI)][conversationallanguage_pypi_package] | [API reference documentation][api_reference_documentation] | [Product documentation][conversationallanguage_docs] | [Samples][conversationallanguage_samples]
 
@@ -32,7 +31,7 @@ pip install azure-ai-language-conversations
 ```
 
 ### Authenticate the client
-In order to interact with the CLU service, you'll need to create an instance of the [ConversationAnalysisClient][conversationanalysis_client_class] class. You will need an **endpoint**, and an **API key** to instantiate a client object. For more information regarding authenticating with Cognitive Services, see [Authenticate requests to Azure Cognitive Services][cognitive_auth].
+In order to interact with the CLU service, you'll need to create an instance of the [ConversationAnalysisClient][conversationanalysis_client_class] class, or [ConversationAuthoringClient][conversationauthoring_client_class] class. You will need an **endpoint**, and an **API key** to instantiate a client object. For more information regarding authenticating with Cognitive Services, see [Authenticate requests to Azure Cognitive Services][cognitive_auth].
 
 #### Get an API key
 You can get the **endpoint** and an **API key** from the Cognitive Services resource in the [Azure Portal][azure_portal].
@@ -56,12 +55,27 @@ credential = AzureKeyCredential("<api-key>")
 client = ConversationAnalysisClient(endpoint, credential)
 ```
 
+#### Create ConversationAuthoringClient
+Once you've determined your **endpoint** and **API key** you can instantiate a `ConversationAuthoringClient`:
+
+```python
+from azure.core.credentials import AzureKeyCredential
+from azure.ai.language.conversations.authoring import ConversationAuthoringClient
+
+endpoint = "https://<my-custom-subdomain>.cognitiveservices.azure.com/"
+credential = AzureKeyCredential("<api-key>")
+client = ConversationAuthoringClient(endpoint, credential)
+```
+
 
 ## Key concepts
 
 ### ConversationAnalysisClient
 The [ConversationAnalysisClient][conversationanalysis_client_class] is the primary interface for making predictions using your deployed Conversations models. For asynchronous operations, an async `ConversationAnalysisClient` is in the `azure.ai.language.conversation.aio` namespace.
 
+### ConversationAnalysisClient
+You can use the [ConversationAuthoringClient][conversationauthoring_client_class] to interface with the [Azure Language Portal][azure_language_portal] to carry out authoring operations on your language resource/project. For example, you can use it to create a project, populate with training data, train, test, and deploy. For asynchronous operations, an async `ConversationAnalysisClient` is in the `azure.ai.language.conversation.authoring.aio` namespace.
+
 ## Examples
 The `azure-ai-language-conversation` client library provides both synchronous and asynchronous APIs.
 
@@ -134,7 +148,6 @@ for entity in result["result"]["prediction"]["entities"]:
                 print("key: {}".format(data["key"]))
             if data["extraInformationKind"] == "EntitySubtype":
                 print("value: {}".format(data["value"]))
-
 ```
 
 ### Analyze Text with an Orchestration App
@@ -197,198 +210,65 @@ if top_intent_object["targetProjectKind"] == "Luis":
     print("\nentities:")
     for entity in luis_response["entities"]:
         print("\n{}".format(entity))
-
 ```
 
-### Conversational Summarization
-
-You can use this sample if you need to summarize a conversation in the form of an issue, and final resolution. For example, a dialog from tech support:
+### Import a Conversation Project
+This sample shows a common scenario for the authoring part of the SDK
 
 ```python
-# import libraries
 import os
 from azure.core.credentials import AzureKeyCredential
+from azure.ai.language.conversations.authoring import ConversationAuthoringClient
 
-from azure.ai.language.conversations import ConversationAnalysisClient
-
-# get secrets
-endpoint = os.environ["AZURE_CONVERSATIONS_ENDPOINT"]
-key = os.environ["AZURE_CONVERSATIONS_KEY"]
-
-# analyze quey
-client = ConversationAnalysisClient(endpoint, AzureKeyCredential(key))
-with client:
-    poller = client.begin_conversation_analysis(
-        task={
-            "displayName": "Analyze conversations from xxx",
-            "analysisInput": {
-                "conversations": [
-                    {
-                        "conversationItems": [
-                            {
-                                "text": "Hello, how can I help you?",
-                                "modality": "text",
-                                "id": "1",
-                                "participantId": "Agent"
-                            },
-                            {
-                                "text": "How to upgrade Office? I am getting error messages the whole day.",
-                                "modality": "text",
-                                "id": "2",
-                                "participantId": "Customer"
-                            },
-                            {
-                                "text": "Press the upgrade button please. Then sign in and follow the instructions.",
-                                "modality": "text",
-                                "id": "3",
-                                "participantId": "Agent"
-                            }
-                        ],
-                        "modality": "text",
-                        "id": "conversation1",
-                        "language": "en"
-                    },
-                ]
-            },
-            "tasks": [
-                {
-                    "taskName": "analyze 1",
-                    "kind": "ConversationalSummarizationTask",
-                    "parameters": {
-                        "summaryAspects": ["Issue, Resolution"]
-                    }
-                }
-            ]
-        }
-    )
+clu_endpoint = os.environ["AZURE_CONVERSATIONS_ENDPOINT"]
+clu_key = os.environ["AZURE_CONVERSATIONS_KEY"]
 
-    # view result
-    result = poller.result()
-    task_result = result["tasks"]["items"][0]
-    print("... view task status ...")
-    print("status: {}".format(task_result["status"]))
-    resolution_result = task_result["results"]
-    if resolution_result["errors"]:
-        print("... errors occured ...")
-        for error in resolution_result["errors"]:
-            print(error)
-    else:
-        conversation_result = resolution_result["conversations"][0]
-        if conversation_result["warnings"]:
-            print("... view warnings ...")
-            for warning in conversation_result["warnings"]:
-                print(warning)
-        else:
-            summaries = conversation_result["summaries"]
-            print("... view task result ...")
-            print("issue: {}".format(summaries[0]["text"]))
-            print("resolution: {}".format(summaries[1]["text"]))
+project_name = "test_project"
+
+exported_project_assets = {
+    "projectKind": "Conversation",
+    "intents": [{"category": "Read"}, {"category": "Delete"}],
+    "entities": [{"category": "Sender"}],
+    "utterances": [
+        {
+            "text": "Open Blake's email",
+            "dataset": "Train",
+            "intent": "Read",
+            "entities": [{"category": "Sender", "offset": 5, "length": 5}],
+        },
+        {
+            "text": "Delete last email",
+            "language": "en-gb",
+            "dataset": "Test",
+            "intent": "Delete",
+            "entities": [],
+        },
+    ],
+}
+
+client = ConversationAuthoringClient(
+    clu_endpoint, AzureKeyCredential(clu_key)
+)
+poller = client.begin_import_project(
+    project_name=project_name,
+    project={
+        "assets": exported_project_assets,
+        "metadata": {
+            "projectKind": "Conversation",
+            "settings": {"confidenceThreshold": 0.7},
+            "projectName": "EmailApp",
+            "multilingual": True,
+            "description": "Trying out CLU",
+            "language": "en-us",
+        },
+        "projectFileVersion": "2022-05-01",
+    },
+)
+response = poller.result()
+print(response)
 
 ```
 
-### Conversational PII
-
-You can use this sample if you need to extract and redact pii info from/in conversations
-
-```python
-# import libraries
-import os
-from azure.core.credentials import AzureKeyCredential
-
-from azure.ai.language.conversations import ConversationAnalysisClient
-
-# get secrets
-endpoint = os.environ["AZURE_CONVERSATIONS_ENDPOINT"]
-key = os.environ["AZURE_CONVERSATIONS_KEY"]
-
-# analyze quey
-client = ConversationAnalysisClient(endpoint, AzureKeyCredential(key))
-with client:
-
-    poller = client.begin_conversation_analysis(
-        task={
-            "displayName": "Analyze PII in conversation",
-            "analysisInput": {
-                "conversations": [
-                    {
-                        "conversationItems": [
-                            {
-                                "id": "1",
-                                "participantId": "0",
-                                "modality": "transcript",
-                                "text": "It is john doe.",
-                                "lexical": "It is john doe",
-                                "itn": "It is john doe",
-                                "maskedItn": "It is john doe"
-                            },
-                            {
-                                "id": "2",
-                                "participantId": "1",
-                                "modality": "transcript",
-                                "text": "Yes, 633-27-8199 is my phone",
-                                "lexical": "yes six three three two seven eight one nine nine is my phone",
-                                "itn": "yes 633278199 is my phone",
-                                "maskedItn": "yes 633278199 is my phone",
-                            },
-                            {
-                                "id": "3",
-                                "participantId": "1",
-                                "modality": "transcript",
-                                "text": "[email protected] is my email",
-                                "lexical": "j dot doe at yahoo dot com is my email",
-                                "maskedItn": "[email protected] is my email",
-                                "itn": "[email protected] is my email",
-                            }
-                        ],
-                        "modality": "transcript",
-                        "id": "1",
-                        "language": "en"
-                    }
-                ]
-            },
-            "tasks": [
-                {
-                    "kind": "ConversationalPIITask",
-                    "parameters": {
-                        "redactionSource": "lexical",
-                        "piiCategories": [
-                            "all"
-                        ]
-                    }
-                }
-            ]
-        }
-    )
-
-    # view result
-    result = poller.result()
-    task_result = result["tasks"]["items"][0]
-    print("... view task status ...")
-    print("status: {}".format(task_result["status"]))
-    conv_pii_result = task_result["results"]
-    if conv_pii_result["errors"]:
-        print("... errors occured ...")
-        for error in conv_pii_result["errors"]:
-            print(error)
-    else:
-        conversation_result = conv_pii_result["conversations"][0]
-        if conversation_result["warnings"]:
-            print("... view warnings ...")
-            for warning in conversation_result["warnings"]:
-                print(warning)
-        else:
-            print("... view task result ...")
-            for conversation in conversation_result["conversationItems"]:
-                print("conversation id: {}".format(conversation["id"]))
-                print("... entities ...")
-                for entity in conversation["entities"]:
-                    print("text: {}".format(entity["text"]))
-                    print("category: {}".format(entity["category"]))
-                    print("confidence: {}".format(entity["confidenceScore"]))
-                    print("offset: {}".format(entity["offset"]))
-                    print("length: {}".format(entity["length"]))
-
-```
 
 ## Optional Configuration
 
@@ -475,5 +355,7 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
 [conversationallanguage_docs]: https://docs.microsoft.com/azure/cognitive-services/language-service/conversational-language-understanding/overview
 [conversationallanguage_samples]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/cognitivelanguage/azure-ai-language-conversations/samples/README.md
 [conversationanalysis_client_class]: https://azuresdkdocs.blob.core.windows.net/$web/python/azure-ai-language-conversations/latest/azure.ai.language.conversations.html#azure.ai.language.conversations.ConversationAnalysisClient
+[conversationauthoring_client_class]: https://azuresdkdocs.blob.core.windows.net/$web/python/azure-ai-language-conversations/latest/azure.ai.language.conversations.html#azure.ai.language.conversations.ConversationAuthoringClient
 [azure_core_exceptions]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md
+[azure_language_portal]: https://language.cognitive.azure.com/home
 ![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-python%2Fsdk%2Ftemplate%2Fazure-template%2FREADME.png)

sdk/cognitivelanguage/azure-ai-language-conversations/azure/ai/language/conversations/_client.py

@@ -9,20 +9,19 @@
 from copy import deepcopy
 from typing import Any, TYPE_CHECKING
 
-from msrest import Deserializer, Serializer
-
 from azure.core import PipelineClient
 from azure.core.credentials import AzureKeyCredential
 from azure.core.rest import HttpRequest, HttpResponse
 
 from ._configuration import ConversationAnalysisClientConfiguration
 from ._operations import ConversationAnalysisClientOperationsMixin
+from ._serialization import Deserializer, Serializer
 
 if TYPE_CHECKING:
     # pylint: disable=unused-import,ungrouped-imports
     from typing import Dict
 
-class ConversationAnalysisClient(ConversationAnalysisClientOperationsMixin):
+class ConversationAnalysisClient(ConversationAnalysisClientOperationsMixin):  # pylint: disable=client-accepts-api-version-keyword
     """The language service conversations API is a suite of natural language processing (NLP) skills
     that can be used to analyze structured conversations (textual or spoken). The synchronous API
     in this suite accepts a request and mediates among multiple language projects, such as LUIS
@@ -35,15 +34,13 @@ class ConversationAnalysisClient(ConversationAnalysisClientOperationsMixin):
     Summarization and Conversational PII detection.
 
     :param endpoint: Supported Cognitive Services endpoint (e.g.,
-     https://:code:`<resource-name>`.api.cognitiveservices.azure.com).
+     https://:code:`<resource-name>`.cognitiveservices.azure.com). Required.
     :type endpoint: str
-    :param credential: Credential needed for the client to connect to Azure.
+    :param credential: Credential needed for the client to connect to Azure. Required.
     :type credential: ~azure.core.credentials.AzureKeyCredential
-    :keyword api_version: Api Version. Default value is "2022-05-15-preview". Note that overriding
-     this default value may result in unsupported behavior.
+    :keyword api_version: Api Version. Default value is "2022-05-01". Note that overriding this
+     default value may result in unsupported behavior.
     :paramtype api_version: str
-    :keyword int polling_interval: Default waiting time between two polls for LRO operations if no
-     Retry-After header is present.
     """
 
     def __init__(
@@ -74,7 +71,7 @@ def send_request(
         >>> response = client.send_request(request)
         <HttpResponse: 200 OK>
 
-        For more information on this code flow, see https://aka.ms/azsdk/python/protocol/quickstart
+        For more information on this code flow, see https://aka.ms/azsdk/dpcodegen/python/send_request
 
         :param request: The network request you want to make. Required.
         :type request: ~azure.core.rest.HttpRequest

sdk/cognitivelanguage/azure-ai-language-conversations/azure/ai/language/conversations/_configuration.py

@@ -22,12 +22,12 @@ class ConversationAnalysisClientConfiguration(Configuration):  # pylint: disable
     attributes.
 
     :param endpoint: Supported Cognitive Services endpoint (e.g.,
-     https://:code:`<resource-name>`.api.cognitiveservices.azure.com).
+     https://:code:`<resource-name>`.cognitiveservices.azure.com). Required.
     :type endpoint: str
-    :param credential: Credential needed for the client to connect to Azure.
+    :param credential: Credential needed for the client to connect to Azure. Required.
     :type credential: ~azure.core.credentials.AzureKeyCredential
-    :keyword api_version: Api Version. Default value is "2022-05-15-preview". Note that overriding
-     this default value may result in unsupported behavior.
+    :keyword api_version: Api Version. Default value is "2022-05-01". Note that overriding this
+     default value may result in unsupported behavior.
     :paramtype api_version: str
     """
 
@@ -38,7 +38,7 @@ def __init__(
         **kwargs: Any
     ) -> None:
         super(ConversationAnalysisClientConfiguration, self).__init__(**kwargs)
-        api_version = kwargs.pop('api_version', "2022-05-15-preview")  # type: str
+        api_version = kwargs.pop('api_version', "2022-05-01")  # type: str
 
         if endpoint is None:
             raise ValueError("Parameter 'endpoint' must not be None.")