Merge pull request #274975 from MicrosoftDocs/main

Publish to live, Friday 4 AM PST, 5/10

Author: ttorble

articles/ai-studio/how-to/prompt-flow-tools/llm-tool.md

@@ -17,14 +17,51 @@ author: lgayhardt
 
 [!INCLUDE [Azure AI Studio preview](../../includes/preview-ai-studio.md)]
 
-To use large language models (LLMs) for natural language processing, you use the prompt flow LLM tool.
+The large language model (LLM) tool in prompt flow enables you to take advantage of widely used large language models like [OpenAI](https://platform.openai.com/), [Azure OpenAI Service](../../../ai-services/openai/overview.md), and models in [Azure AI Studio model catalog](../model-catalog.md) for natural language processing.
+> [!NOTE]
+> The previous version of the LLM tool is now being deprecated. Please upgrade to latest [promptflow-tools](https://pypi.org/project/promptflow-tools/) package to consume new llm tools.
+
+Prompt flow provides a few different large language model APIs:
+
+- [Completion](https://platform.openai.com/docs/api-reference/completions): OpenAI's completion models generate text based on provided prompts.
+- [Chat](https://platform.openai.com/docs/api-reference/chat): OpenAI's chat models facilitate interactive conversations with text-based inputs and responses.
 
 > [!NOTE]
-> For embeddings to convert text into dense vector representations for various natural language processing tasks, see [Embedding tool](embedding-tool.md).
+> Don't use non-ascii characters in resource group name of Azure OpenAI resource, prompt flow didn't support this case.
 
 ## Prerequisites
 
-Prepare a prompt as described in the [Prompt tool](prompt-tool.md#prerequisites) documentation. The LLM tool and Prompt tool both support [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) templates. For more information and best practices, see [Prompt engineering techniques](../../../ai-services/openai/concepts/advanced-prompt-engineering.md).
+Create OpenAI resources, Azure OpenAI resources, or MaaS deployment with the LLM models (for example: llama2, mistral, cohere etc.) in Azure AI Studio model catalog:
+
+- **OpenAI**:
+
+    - Sign up your account on the [OpenAI website](https://openai.com/).
+    - Sign in and [find your personal API key](https://platform.openai.com/account/api-keys).
+
+- **Azure OpenAI**:
+
+    - [Create Azure OpenAI resources](../../../ai-services/openai/how-to/create-resource.md).
+
+- **MaaS deployment**:
+
+    [Create MaaS deployment for models in Azure AI Studio model catalog](../../concepts/deployments-overview.md#deploy-models-with-model-as-a-service).
+
+    You can create serverless connection to use this MaaS deployment.
+
+## Connections
+
+Set up connections to provisioned resources in prompt flow.
+
+| Type        | Name     | API key  | API base | API type | API version |
+|-------------|----------|----------|----------|----------|-------------|
+| OpenAI      | Required | Required | -        | -        | -           |
+| Azure OpenAI| Required | Required | Required | Required | Required    |
+| Serverless  | Required | Required | Required | -        | -           |
+
+  > [!TIP]
+  > - To use Microsoft Entra ID auth type for Azure OpenAI connection, you need assign either the `Cognitive Services OpenAI User` or `Cognitive Services OpenAI Contributor role` to user or user assigned managed identity.
+  > - Learn more about [how to specify to use user identity to submit flow run](../create-manage-runtime.md#create-an-automatic-runtime-on-a-flow-page).
+  > - Learn more about [How to configure Azure OpenAI Service with managed identities](../../../ai-services/openai/how-to/managed-identity.md).
 
 ## Build with the LLM tool
 
@@ -35,7 +72,7 @@ Prepare a prompt as described in the [Prompt tool](prompt-tool.md#prerequisites)
 
 1. Select the connection to one of your provisioned resources. For example, select **Default_AzureOpenAI**.
 1. From the **Api** dropdown list, select **chat** or **completion**.
-1. Enter values for the LLM tool input parameters described in the [Text completion inputs table](#inputs). If you selected the **chat** API, see the [Chat inputs table](#chat-inputs). If you selected the **completion** API, see the [Text completion inputs table](#text-completion-inputs). For information about how to prepare the prompt input, see [Prerequisites](#prerequisites).
+1. Enter values for the LLM tool input parameters described in the [Text completion inputs table](#inputs). If you selected the **chat** API, see the [Chat inputs table](#chat-inputs). If you selected the **completion** API, see the [Text completion inputs table](#text-completion-inputs). For information about how to prepare the prompt input, see [How to write a prompt](#how-to-write-a-prompt).
 1. Add more tools to your flow, as needed. Or select **Run** to run the flow.
 1. The outputs are described in the [Outputs table](#outputs).
 
@@ -74,15 +111,146 @@ The following input parameters are available.
 | presence\_penalty      | float       | The value that controls the model's behavior regarding repeating phrases. Default is 0.      | No       |
 | frequency\_penalty     | float       | The value that controls the model's behavior regarding generating rare phrases. Default is 0. | No       |
 | logit\_bias            | dictionary  | The logit bias for the language model. Default is empty dictionary.                            | No       |
+| tool\_choice           | object      | Value that controls which tool is called by the model. Default is null.                        | No       |
+| tools                  | list        | A list of tools the model may generate JSON inputs for. Default is null.                       | No       |
+| response_format        | object      | An object specifying the format that the model must output. Default is null.                   | No       |
 
 ## Outputs
 
 The output varies depending on the API you selected for inputs.
 
-| API        | Return type | Description                              |
-|------------|-------------|------------------------------------------|
-| Completion | string      | The text of one predicted completion.     |
-| Chat       | string      | The text of one response of conversation. |
+| Return type | Description                              |
+|-------------|------------------------------------------|
+| string      | Text of one predicted completion or response of conversation |
+
+## How to write a prompt?
+
+Prepare a prompt as described in the [Prompt tool](prompt-tool.md#prerequisites) documentation. The LLM tool and Prompt tool both support [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) templates. For more information and best practices, see [Prompt engineering techniques](../../../ai-services/openai/concepts/advanced-prompt-engineering.md).
+
+For example, for a chat prompt we offer a method to distinguish between different roles in a chat prompt, such as "system", "user", "assistant" and "tool". The "system", "user", "assistant" roles can have "name" and "content" properties. The "tool" role, however, should have "tool_call_id" and "content" properties. For an example of a tool chat prompt, please refer to [Sample 3](#sample-3).
+
+### Sample 1
+```jinja
+# system:
+You are a helpful assistant.
+
+{% for item in chat_history %}
+# user:
+{{item.inputs.question}}
+# assistant:
+{{item.outputs.answer}}
+{% endfor %}
+
+# user:
+{{question}}
+```
+
+In LLM tool, the prompt is transformed to match the [OpenAI messages](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages) structure before sending to OpenAI chat API.
+
+```
+[
+    {
+        "role": "system",
+        "content": "You are a helpful assistant."
+    },
+    {
+        "role": "user",
+        "content": "<question-of-chat-history-round-1>"
+    },
+    {
+        "role": "assistant",
+        "content": "<answer-of-chat-history-round-1>"
+    },
+    ...
+    {
+        "role": "user",
+        "content": "<question>"
+    }
+]
+```
+
+### Sample 2
+```jinja
+# system:
+{# For role naming customization, the following syntax is used #}
+## name:
+Alice
+## content:
+You are a bot can tell good jokes.
+```
+
+In LLM tool, the prompt is transformed to match the [OpenAI messages](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages) structure before sending to OpenAI chat API.
+
+```
+[
+    {
+        "role": "system",
+        "name": "Alice",
+        "content": "You are a bot can tell good jokes."
+    }
+]
+```
+
+### Sample 3
+This sample illustrates how to write a tool chat prompt.
+```jinja
+# system:
+You are a helpful assistant.
+# user:
+What is the current weather like in Boston?
+# assistant:
+{# The assistant message with 'tool_calls' must be followed by messages with role 'tool'. #}
+## tool_calls:
+{{llm_output.tool_calls}}
+# tool:
+{#
+Messages with role 'tool' must be a response to a preceding message with 'tool_calls'.
+Additionally, 'tool_call_id's should match ids of assistant message 'tool_calls'.
+#}
+## tool_call_id:
+{{llm_output.tool_calls[0].id}}
+## content:
+{{tool-answer-of-last-question}}
+# user:
+{{question}}
+```
+
+In LLM tool, the prompt is transformed to match the [OpenAI messages](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages) structure before sending to OpenAI chat API.
+
+```
+[
+    {
+        "role": "system",
+        "content": "You are a helpful assistant."
+    },
+    {
+        "role": "user",
+        "content": "What is the current weather like in Boston?"
+    },
+    {
+        "role": "assistant",
+        "content": null,
+        "function_call": null,
+        "tool_calls": [
+            {
+                "id": "<tool-call-id-of-last-question>",
+                "type": "function",
+                "function": "<function-to-call-of-last-question>"
+            }
+        ]
+    },
+    {
+        "role": "tool",
+        "tool_call_id": "<tool-call-id-of-last-question>",
+        "content": "<tool-answer-of-last-question>"
+    }
+    ...
+    {
+        "role": "user",
+        "content": "<question>"
+    }
+]
+```
 
 ## Next steps
 

articles/ai-studio/how-to/prompt-flow-tools/prompt-flow-tools-overview.md

@@ -19,7 +19,7 @@ The following table provides an index of tools in prompt flow.
 
 | Tool name | Description | Package name |
 |------|-----------|-------------|--------------|
-| [LLM](./llm-tool.md) | Use large language models (LLM) with the Azure OpenAI Service for tasks such as text completion or chat. | [promptflow-tools](https://pypi.org/project/promptflow-tools/) |
+| [LLM](./llm-tool.md) | Use large language models (LLM) for tasks such as text completion or chat. | [promptflow-tools](https://pypi.org/project/promptflow-tools/) |
 | [Prompt](./prompt-tool.md) | Craft a prompt by using Jinja as the templating language. | [promptflow-tools](https://pypi.org/project/promptflow-tools/) |
 | [Python](./python-tool.md) | Run Python code. | [promptflow-tools](https://pypi.org/project/promptflow-tools/) |
 | [Azure OpenAI GPT-4 Turbo with Vision](./azure-open-ai-gpt-4v-tool.md) | Use an Azure OpenAI GPT-4 Turbo with Vision model deployment to analyze images and provide textual responses to questions about them. | [promptflow-tools](https://pypi.org/project/promptflow-tools/) |

articles/ai-studio/media/prompt-flow/llm-tool.png

@@ -3,7 +3,7 @@ title: How to use SignalR Service with Azure Application Gateway
 description: This article provides information about using Azure SignalR Service with Azure Application Gateway.
 author: vicancy
 ms.author: lianwei
-ms.date: 08/16/2022
+ms.date: 05/10/2024
 ms.service: signalr
 ms.topic: how-to
 ---
@@ -241,24 +241,26 @@ Let's deploy the Chat application into the same VNet with **_ASRS1_** so that th
 
 ### Deploy the chat application to Azure
 
-- On the [Azure portal](https://portal.azure.com/), search for **App services** and **Create**.
+- On the [Azure portal](https://portal.azure.com/), search for **App services** and **Create** **Web App**.
 
-- On the **Basics** tab, use these values for the following application gateway settings:
+- On the **Basics** tab, use these values for the following web app settings:
   - **Subscription** and **Resource group** and **Region**: the same as what you choose for SignalR Service
   - **Name**: **_WA1_**
   * **Publish**: **Code**
   * **Runtime stack**: **.NET 6 (LTS)**
   * **Operating System**: **Linux**
   * **Region**: Make sure it's the same as what you choose for SignalR Service
-  * Select **Next: Docker**
+  * Select **Next: Deployment**, keep all as default, and select **Next:Networking**
 - On the **Networking** tab
   - **Enable network injection**: select **On**
   - **Virtual Network**: select **_VN1_** we previously created
   - **Enable VNet integration**: **On**
   - **Outbound subnet**: create a new subnet
   - Select **Review + create**
 
-Now let's deploy our chat application to Azure. Below we use Azure CLI to deploy the web app, you can also choose other deployment environments following [publish your web app section](/azure/app-service/quickstart-dotnetcore#publish-your-web-app).
+Now let's deploy our chat application to Azure. Below 
+
+We use Azure CLI to deploy our chat application to Azure. Check [Quickstart: Deploy an ASP.NET web app](/azure/app-service/quickstart-dotnetcore) for other deployment environments deploying to Azure.
 
 Under folder samples/Chatroom, run the below commands:
 
@@ -271,7 +273,7 @@ zip -r app.zip .
 # use az CLI to deploy app.zip to our webapp
 az login
 az account set -s <your-subscription-name-used-to-create-WA1>
-az webapp deployment source config-zip -n WA1 -g <resource-group-of-WA1> --src app.zip
+az webapp deploy -g <resource-group-of-WA1> -n WA1 --src-path app.zip
 ```
 
 Now the web app is deployed, let's go to the portal for **_WA1_** and make the following updates:

articles/azure-signalr/signalr-howto-work-with-app-gateway.md

@@ -651,6 +651,8 @@ items:
         href: ./hadoop/troubleshoot-wasbs-storage-exception.md
       - name: Manage disk space
         href: ./hadoop/troubleshoot-disk-space.md
+      - name: Ambari user configs migration
+        href: ./migrate-ambari-recent-version-hdinsight.md  
 - name: Apache Kafka
   items:
   - name: Overview