You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-studio/how-to/prompt-flow-tools/llm-tool.md
+8-176Lines changed: 8 additions & 176 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,51 +17,14 @@ author: lgayhardt
17
17
18
18
[!INCLUDE [Azure AI Studio preview](../../includes/preview-ai-studio.md)]
19
19
20
-
The large language model (LLM) tool in prompt flow enables you to take advantage of widely used large language models like [OpenAI](https://platform.openai.com/), [Azure OpenAI Service](../../../ai-services/openai/overview.md), and models in [Azure AI Studio model catalog](../model-catalog.md) for natural language processing.
21
-
> [!NOTE]
22
-
> The previous version of the LLM tool is now being deprecated. Please upgrade to latest [promptflow-tools](https://pypi.org/project/promptflow-tools/) package to consume new llm tools.
23
-
24
-
Prompt flow provides a few different large language model APIs:
25
-
26
-
-[Completion](https://platform.openai.com/docs/api-reference/completions): OpenAI's completion models generate text based on provided prompts.
27
-
-[Chat](https://platform.openai.com/docs/api-reference/chat): OpenAI's chat models facilitate interactive conversations with text-based inputs and responses.
20
+
To use large language models (LLMs) for natural language processing, you use the prompt flow LLM tool.
28
21
29
22
> [!NOTE]
30
-
> Don't use non-ascii characters in resource group name of Azure OpenAI resource, prompt flow didn't support this case.
23
+
> For embeddings to convert text into dense vector representations for various natural language processing tasks, see [Embedding tool](embedding-tool.md).
31
24
32
25
## Prerequisites
33
26
34
-
Create OpenAI resources, Azure OpenAI resources, or MaaS deployment with the LLM models (for example: llama2, mistral, cohere etc.) in Azure AI Studio model catalog:
35
-
36
-
-**OpenAI**:
37
-
38
-
- Sign up your account on the [OpenAI website](https://openai.com/).
39
-
- Sign in and [find your personal API key](https://platform.openai.com/account/api-keys).
> - To use Microsoft Entra ID auth type for Azure OpenAI connection, you need assign either the `Cognitive Services OpenAI User` or `Cognitive Services OpenAI Contributor role` to user or user assigned managed identity.
63
-
> - Learn more about [how to specify to use user identity to submit flow run](../create-manage-runtime.md#create-an-automatic-runtime-on-a-flow-page).
64
-
> - Learn more about [How to configure Azure OpenAI Service with managed identities](../../../ai-services/openai/how-to/managed-identity.md).
27
+
Prepare a prompt as described in the [Prompt tool](prompt-tool.md#prerequisites) documentation. The LLM tool and Prompt tool both support [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) templates. For more information and best practices, see [Prompt engineering techniques](../../../ai-services/openai/concepts/advanced-prompt-engineering.md).
65
28
66
29
## Build with the LLM tool
67
30
@@ -72,7 +35,7 @@ Set up connections to provisioned resources in prompt flow.
72
35
73
36
1. Select the connection to one of your provisioned resources. For example, select **Default_AzureOpenAI**.
74
37
1. From the **Api** dropdown list, select **chat** or **completion**.
75
-
1. Enter values for the LLM tool input parameters described in the [Text completion inputs table](#inputs). If you selected the **chat** API, see the [Chat inputs table](#chat-inputs). If you selected the **completion** API, see the [Text completion inputs table](#text-completion-inputs). For information about how to prepare the prompt input, see [How to write a prompt](#how-to-write-a-prompt).
38
+
1. Enter values for the LLM tool input parameters described in the [Text completion inputs table](#inputs). If you selected the **chat** API, see the [Chat inputs table](#chat-inputs). If you selected the **completion** API, see the [Text completion inputs table](#text-completion-inputs). For information about how to prepare the prompt input, see [Prerequisites](#prerequisites).
76
39
1. Add more tools to your flow, as needed. Or select **Run** to run the flow.
77
40
1. The outputs are described in the [Outputs table](#outputs).
78
41
@@ -111,146 +74,15 @@ The following input parameters are available.
111
74
| presence\_penalty | float | The value that controls the model's behavior regarding repeating phrases. Default is 0. | No |
112
75
| frequency\_penalty | float | The value that controls the model's behavior regarding generating rare phrases. Default is 0. | No |
113
76
| logit\_bias | dictionary | The logit bias for the language model. Default is empty dictionary. | No |
114
-
| tool\_choice | object | Value that controls which tool is called by the model. Default is null. | No |
115
-
| tools | list | A list of tools the model may generate JSON inputs for. Default is null. | No |
116
-
| response_format | object | An object specifying the format that the model must output. Default is null. | No |
117
77
118
78
## Outputs
119
79
120
80
The output varies depending on the API you selected for inputs.
| string | Text of one predicted completion or response of conversation |
125
-
126
-
## How to write a prompt?
127
-
128
-
Prepare a prompt as described in the [Prompt tool](prompt-tool.md#prerequisites) documentation. The LLM tool and Prompt tool both support [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) templates. For more information and best practices, see [Prompt engineering techniques](../../../ai-services/openai/concepts/advanced-prompt-engineering.md).
129
-
130
-
For example, for a chat prompt we offer a method to distinguish between different roles in a chat prompt, such as "system", "user", "assistant" and "tool". The "system", "user", "assistant" roles can have "name" and "content" properties. The "tool" role, however, should have "tool_call_id" and "content" properties. For an example of a tool chat prompt, please refer to [Sample 3](#sample-3).
131
-
132
-
### Sample 1
133
-
```jinja
134
-
# system:
135
-
You are a helpful assistant.
136
-
137
-
{% for item in chat_history %}
138
-
# user:
139
-
{{item.inputs.question}}
140
-
# assistant:
141
-
{{item.outputs.answer}}
142
-
{% endfor %}
143
-
144
-
# user:
145
-
{{question}}
146
-
```
147
-
148
-
In LLM tool, the prompt is transformed to match the [OpenAI messages](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages) structure before sending to OpenAI chat API.
149
-
150
-
```
151
-
[
152
-
{
153
-
"role": "system",
154
-
"content": "You are a helpful assistant."
155
-
},
156
-
{
157
-
"role": "user",
158
-
"content": "<question-of-chat-history-round-1>"
159
-
},
160
-
{
161
-
"role": "assistant",
162
-
"content": "<answer-of-chat-history-round-1>"
163
-
},
164
-
...
165
-
{
166
-
"role": "user",
167
-
"content": "<question>"
168
-
}
169
-
]
170
-
```
171
-
172
-
### Sample 2
173
-
```jinja
174
-
# system:
175
-
{# For role naming customization, the following syntax is used #}
176
-
## name:
177
-
Alice
178
-
## content:
179
-
You are a bot can tell good jokes.
180
-
```
181
-
182
-
In LLM tool, the prompt is transformed to match the [OpenAI messages](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages) structure before sending to OpenAI chat API.
183
-
184
-
```
185
-
[
186
-
{
187
-
"role": "system",
188
-
"name": "Alice",
189
-
"content": "You are a bot can tell good jokes."
190
-
}
191
-
]
192
-
```
193
-
194
-
### Sample 3
195
-
This sample illustrates how to write a tool chat prompt.
196
-
```jinja
197
-
# system:
198
-
You are a helpful assistant.
199
-
# user:
200
-
What is the current weather like in Boston?
201
-
# assistant:
202
-
{# The assistant message with 'tool_calls' must be followed by messages with role 'tool'. #}
203
-
## tool_calls:
204
-
{{llm_output.tool_calls}}
205
-
# tool:
206
-
{#
207
-
Messages with role 'tool' must be a response to a preceding message with 'tool_calls'.
208
-
Additionally, 'tool_call_id's should match ids of assistant message 'tool_calls'.
209
-
#}
210
-
## tool_call_id:
211
-
{{llm_output.tool_calls[0].id}}
212
-
## content:
213
-
{{tool-answer-of-last-question}}
214
-
# user:
215
-
{{question}}
216
-
```
217
-
218
-
In LLM tool, the prompt is transformed to match the [OpenAI messages](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages) structure before sending to OpenAI chat API.
219
-
220
-
```
221
-
[
222
-
{
223
-
"role": "system",
224
-
"content": "You are a helpful assistant."
225
-
},
226
-
{
227
-
"role": "user",
228
-
"content": "What is the current weather like in Boston?"
Copy file name to clipboardExpand all lines: articles/ai-studio/how-to/prompt-flow-tools/prompt-flow-tools-overview.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,7 +19,7 @@ The following table provides an index of tools in prompt flow.
19
19
20
20
| Tool name | Description | Package name |
21
21
|------|-----------|-------------|--------------|
22
-
|[LLM](./llm-tool.md)| Use large language models (LLM) for tasks such as text completion or chat. |[promptflow-tools](https://pypi.org/project/promptflow-tools/)|
22
+
|[LLM](./llm-tool.md)| Use large language models (LLM) with the Azure OpenAI Service for tasks such as text completion or chat. |[promptflow-tools](https://pypi.org/project/promptflow-tools/)|
23
23
|[Prompt](./prompt-tool.md)| Craft a prompt by using Jinja as the templating language. |[promptflow-tools](https://pypi.org/project/promptflow-tools/)|
24
24
|[Python](./python-tool.md)| Run Python code. |[promptflow-tools](https://pypi.org/project/promptflow-tools/)|
25
25
|[Azure OpenAI GPT-4 Turbo with Vision](./azure-open-ai-gpt-4v-tool.md)| Use an Azure OpenAI GPT-4 Turbo with Vision model deployment to analyze images and provide textual responses to questions about them. |[promptflow-tools](https://pypi.org/project/promptflow-tools/)|
Copy file name to clipboardExpand all lines: articles/azure-app-configuration/quickstart-bicep.md
+5-2Lines changed: 5 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,6 +28,9 @@ If you don't have an Azure subscription, create a [free account](https://azure.m
28
28
29
29
Managing an Azure App Configuration resource with Bicep file requires an Azure Resource Manager role, such as contributor or owner. Accessing Azure App Configuration data (key-values, snapshots) requires an Azure Resource Manager role and an additional Azure App Configuration [data plane role](concept-enable-rbac.md) when the configuration store's ARM authentication mode is set to [pass-through](./quickstart-deployment-overview.md#azure-resource-manager-authentication-mode) ARM authentication mode.
30
30
31
+
> [!IMPORTANT]
32
+
> Configuring ARM authentication mode requires App Configuration control plane API version `2023-08-01-preview` or later.
33
+
31
34
## Review the Bicep file
32
35
33
36
The Bicep file used in this quickstart is from [Azure Quickstart Templates](https://azure.microsoft.com/resources/templates/app-configuration-store-kv/).
@@ -39,8 +42,8 @@ The Bicep file used in this quickstart is from [Azure Quickstart Templates](http
39
42
40
43
Two Azure resources are defined in the Bicep file:
41
44
42
-
-[Microsoft.AppConfiguration/configurationStores](/azure/templates/microsoft.appconfiguration/2020-07-01-preview/configurationstores): create an App Configuration store.
43
-
-[Microsoft.AppConfiguration/configurationStores/keyValues](/azure/templates/microsoft.appconfiguration/2020-07-01-preview/configurationstores/keyvalues): create a key-value inside the App Configuration store.
45
+
-[Microsoft.AppConfiguration/configurationStores](/azure/templates/microsoft.appconfiguration/configurationstores): create an App Configuration store.
46
+
-[Microsoft.AppConfiguration/configurationStores/keyValues](/azure/templates/microsoft.appconfiguration/configurationstores/keyvalues): create a key-value inside the App Configuration store.
44
47
45
48
With this Bicep file, we create one key with two different values, one of which has a unique label.
Copy file name to clipboardExpand all lines: articles/azure-app-configuration/quickstart-deployment-overview.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -33,6 +33,9 @@ To learn more about Azure RBAC and Microsoft Entra ID, see [Authorize access to
33
33
34
34
Azure App Configuration data, such as key-values and snapshots, can be managed in deployment. When managing App Configuration data using this method, it's recommended to set your configuration store's Azure Resource Manager authentication mode to **Pass-through**. This authentication mode ensures that data access requires a combination of data plane and Azure Resource Manager management roles and ensuring that data access can be properly attributed to the deployment caller for auditing purpose.
35
35
36
+
> [!IMPORTANT]
37
+
> App Configuration control plane API version `2023-08-01-preview` or later is required to configure **Azure Resource Manager Authentication Mode** using [ARM template](./quickstart-resource-manager.md), [Bicep](./quickstart-bicep.md), or REST API. See the [REST API examples](https://github.com/Azure/azure-rest-api-specs/blob/main/specification/appconfiguration/resource-manager/Microsoft.AppConfiguration/preview/2023-08-01-preview/examples/ConfigurationStoresCreateWithDataPlaneProxy.json).
Copy file name to clipboardExpand all lines: articles/azure-app-configuration/quickstart-resource-manager.md
+6-3Lines changed: 6 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -35,21 +35,24 @@ If you don't have an Azure subscription, create a [free account](https://azure.m
35
35
36
36
Managing Azure App Configuration resource inside an ARM template requires Azure Resource Manager role, such as contributor or owner. Accessing Azure App Configuration data (key-values, snapshots) requires Azure Resource Manager role and Azure App Configuration [data plane role](concept-enable-rbac.md) under [pass-through](./quickstart-deployment-overview.md#azure-resource-manager-authentication-mode) ARM authentication mode.
37
37
38
+
> [!IMPORTANT]
39
+
> Configuring ARM authentication mode requires App Configuration control plane API version `2023-08-01-preview` or later.
40
+
38
41
## Review the template
39
42
40
43
The template used in this quickstart is from [Azure Quickstart Templates](https://azure.microsoft.com/resources/templates/app-configuration-store-kv/). It creates a new App Configuration store with two key-values inside. It then uses the `reference` function to output the values of the two key-value resources. Reading the key's value in this way allows it to be used in other places in the template.
41
44
42
45
The quickstart uses the `copy` element to create multiple instances of key-value resource. To learn more about the `copy` element, see [Resource iteration in ARM templates](../azure-resource-manager/templates/copy-resources.md).
43
46
44
47
> [!IMPORTANT]
45
-
> This template requires App Configuration resource provider version `2020-07-01-preview` or later. This version uses the `reference` function to read key-values. The `listKeyValue` function that was used to read key-values in the previous version is not available starting in version `2020-07-01-preview`.
48
+
> This template requires App Configuration control plane API version `2022-05-01` or later. This version uses the `reference` function to read key-values. The `listKeyValue` function that was used to read key-values in the previous version is not available starting in version `2020-07-01-preview`.
-[Microsoft.AppConfiguration/configurationStores](/azure/templates/microsoft.appconfiguration/2020-07-01-preview/configurationstores): create an App Configuration store.
52
-
-[Microsoft.AppConfiguration/configurationStores/keyValues](/azure/templates/microsoft.appconfiguration/2020-07-01-preview/configurationstores/keyvalues): create a key-value inside the App Configuration store.
54
+
-[Microsoft.AppConfiguration/configurationStores](/azure/templates/microsoft.appconfiguration/configurationstores): create an App Configuration store.
55
+
-[Microsoft.AppConfiguration/configurationStores/keyValues](/azure/templates/microsoft.appconfiguration/configurationstores/keyvalues): create a key-value inside the App Configuration store.
53
56
54
57
> [!TIP]
55
58
> The `keyValues` resource's name is a combination of key and label. The key and label are joined by the `$` delimiter. The label is optional. In the above example, the `keyValues` resource with name `myKey` creates a key-value without a label.
0 commit comments